MMSys'21 Challenge Clarifications

198 views
Skip to first unread message

Lim Mei Ling May

unread,
Mar 30, 2021, 5:20:50 PM3/30/21
to opennetlab...@googlegroups.com, Ali C Begen, Roger Zimmermann, Abdelhak Bentaleb, Mehmet Necmettin Akcay

Dear organizers,

We have a few questions regarding the MMSys’21 challenge on “Bandwidth Estimation for Real-Time Communications” -

Q1. AlphaRTC Bandwidth Estimator (Input Data):

Is there a documentation that lists the data points available to the bandwidth estimator and how to retrieve them? The challenge is about the sending traffic, so we are expected to estimate the sending (upload) bandwidth. Should we assume the download speed is never an issue?

Q2. AlphaRTC Bandwidth Estimator (Relation to Congestion Control):

The challenge description says that the output of the bandwidth estimator is the bitrate value fed to the codec for encoding purposes. We are not concerned with the packet-level sending strategy, right (it is taken care of automatically by the GCC which we should not modify at all)?

Q3. Test Scenarios:

Is there more information on the test scenarios/ test cases? E.g., what is the max number of instances in each test case? How will the instances be connecting to each other? If it is two clients talking to each other, they will have a direct connection but if there are more than two, they will likely go through a server (forwarding the copy to each other client). Are we concerned about only the upload (sending) bandwidth of the active client (to the forwarding unit) and not the download speeds of the non-active clients?

 

Thank you

Regards,

May

OpenNetLab Challenge

unread,
Apr 1, 2021, 10:13:32 AM4/1/21
to OpenNetLab Challenge
Hi May,

    Thanks for your attention to our Challenge. 

    Q1. AlphaRTC Bandwidth Estimator (Input Data):

    We will use AlphaRTC as the challenge applicant to evaluate the bandwidth estimator provided by contestant. You can implement a bandwidth estimator with pyinfer( https://github.com/OpenNetLab/AlphaRTC#pyinfer  ). As we mentioned in this Readme, you need to implement two interfaces, report_states and get_estimated_bandwidth. The first interface, report_states, is the input data, it's an abstract of  rtcp/rtp packet. The second interfaces, get_estimated_bandwidth, should return your estimated bandwidth when the AlphaRTC call this interface.
    Actually, the scenario of our challenge is very simple, we will just establish one P2P connection. So you should estimated the unidirectional bandwidth.

    Q2. AlphaRTC Bandwidth Estimator (Relation to Congestion Control):
    
    You don't need to change or care about anything of GCC, Actually, we replace GCC in WebRTC with the outside python estimator pyinfer. You should implement pyinfer like this example (https://github.com/OpenNetLab/AlphaRTC/blob/main/examples/peerconnection/serverless/corpus/BandwidthEstimator.py) We provided a Gym (https://github.com/OpenNetLab/gym) for contestants to train their pyinfer. And here is an very sample example (https://github.com/OpenNetLab/gym-example

    Q3. Test Scenarios:

    As I mentioned at the Q1, our scenario is P2P connection. We will pick a pair of nodes in our Open Net Lab platforms and deploy AlphaRTC and contestants' bandwidth estimator on the two nodes. Of course, we will test the bandwidth estimator many times under different network scenario.

    I hope my answer can help you. If you have more concerns or question, please don't hesitated to contact us.

Thanks,
Ze

Parth Thakkar

unread,
Apr 30, 2021, 3:07:01 AM4/30/21
to OpenNetLab Challenge
Hello,

> As I mentioned at the Q1, our scenario is P2P connection. We will pick a pair of nodes in our Open Net Lab platforms and deploy AlphaRTC and contestants' bandwidth estimator on the two nodes. Of course, we will test the bandwidth estimator many times under different network scenario.

Does this mean there will be no cross-traffic? Or there will be traffic between other nodes concurrent to the P2P connection?

Zhixiong Niu

unread,
Apr 30, 2021, 4:46:20 AM4/30/21
to Parth Thakkar, OpenNetLab Challenge

Hi Parth,

 

Actually, the nodes are located in real networks. Hence, we will have cross-traffic at least by others.

 

We may also test the BWE in cross-traffic intentionally.

 

-- 

Best,

Zhixiong

--
You received this message because you are subscribed to the Google Groups "OpenNetLab Challenge" group.
To unsubscribe from this group and stop receiving emails from it, send an email to opennetlab-chall...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/opennetlab-challenge/be19c1ae-38f9-4012-9aa2-1eef6fe2d551n%40googlegroups.com.

May Lim

unread,
May 4, 2021, 4:39:56 AM5/4/21
to OpenNetLab Challenge
Hi organizers,

In the challenge website, it mentions that “We will also release a limited number of network traces collected in the test environment as examples; stay tuned for further updates on OpenNetLab’s website.”

May I know if these test network traces have been released? (I haven't been able to find them on the links so far)

Thank you.
Regards,
May

Zhixiong Niu

unread,
May 5, 2021, 7:50:24 PM5/5/21
to May Lim, OpenNetLab Challenge

Zhixiong Niu

unread,
May 6, 2021, 1:28:43 AM5/6/21
to May Lim, OpenNetLab Challenge

Hi May,

 

For more traces, we will keep you updated.

 

We created a slack channel for this challenge. If you want to participant this challenge, could you join our Slack channel (https://join.slack.com/t/opennetlab-challenge/shared_invite/zt-pjn74xhx-d~jG5lY3s4_6kSJHuzfHcw)?

 

 

-- 

Best,

Zhixiong

May Lim

unread,
May 20, 2021, 10:37:07 PM5/20/21
to OpenNetLab Challenge
Hi Zhixiong,

Thank you, I have joined the Slack channel.

Another follow-up question - 
Regarding "the first interface, report_states, is the input data, it's an abstract of  rtcp/rtp packet", apart from the input data currently exposed in the code, are we allowed to retrieve other data inputs for our use? (Or does the challenge only allow the use of those data?)

Regards,
May

OpenNetLab Challenge

unread,
May 21, 2021, 2:29:25 AM5/21/21
to OpenNetLab Challenge

Hi May,

    We don't have a short-term plan to support other input data. But it's very welcome if you have any proposal, we need to estimate the workload to decide whether to export more data you proposed to user.

Thanks,
Ze
Reply all
Reply to author
Forward
0 new messages