soheil-ab / c2tcp

C2TCP: A Flexible Cellular TCP to Meet Stringent Delay Requirements.

C2TCP v2.0

C2TCP v2.2: A Flexible Cellular TCP to Meet Stringent Delay Requirements.

()To see the IEEE JSAC'19 release, please checkout branch named v2.0-jsac-19. Also, the source code of C2TCP.v1.0 {https://doi.org/10.23919/IFIPNetworking.2018.8696844 , https://arxiv.org/abs/1807.02689 } published in IFIP Networking Confernece, May 2018, can be accessed on https://github.com/Soheil-ab/C2TCP-IFIP)

Installation Guide

Here we will provide you with detailed instructions to test C2TCP over Mahimahi.

Getting the Source Code:

Note: C2TCP is implemented on Linux kernel 4.13.1.

Get the source code:

cd ~
git clone https://github.com/Soheil-ab/c2tcp.git
cd c2tcp

Installing Required Tools

General Note: Installing any of the following tools/schemes, depending on your machine, might require other libraries (in addition to what have been mentioned here). So, if you get any errors mentioning that something not being found when you try to make, install them using apt.

1. Install Mahimahi (http://mahimahi.mit.edu/#getting)

   cd ~/c2tcp/
   sudo apt-get install build-essential git debhelper autotools-dev dh-autoreconf iptables protobuf-compiler libprotobuf-dev pkg-config libssl-dev dnsmasq-base ssl-cert libxcb-present-dev libcairo2-dev libpango1.0-dev iproute2 apache2-dev apache2-bin iptables dnsmasq-base gnuplot iproute2 apache2-api-20120211 libwww-perl
   git clone https://github.com/ravinet/mahimahi 
   cd mahimahi
   ./autogen.sh && ./configure && make
   sudo make install
   sudo sysctl -w net.ipv4.ip_forward=1

2. Install iperf3

   sudo apt-get remove iperf3 libiperf0
   wget https://iperf.fr/download/ubuntu/libiperf0_3.1.3-1_amd64.deb
   wget https://iperf.fr/download/ubuntu/iperf3_3.1.3-1_amd64.deb
   sudo dpkg -i libiperf0_3.1.3-1_amd64.deb iperf3_3.1.3-1_amd64.deb
   rm libiperf0_3.1.3-1_amd64.deb iperf3_3.1.3-1_amd64.deb

Installing Other Schemes

BBR, C2TCP, Westwood, and Cubic are already part of the the current Kernel patch.

Patching C2TCP Kernel: Install the prepared debian packages.

Simply install the debian packages of the patched kernel:

cd ~/c2tcp/linux-patch
sudo dpkg -i linux-image*521*
sudo dpkg -i linux-header*521*
sudo reboot 

This Kernel version includes DeepCC and Orca schemes too.

Orca: Soheil Abbasloo, et. al. "Classic Meets Modern: A Pragmatic Learning-Based COngestion COntrol for the Internet", In proc. ACM SIGCOMM 2020 https://dl.acm.org/doi/abs/10.1145/3387514.3405892 DeepCC: Soheil Abbasloo, et. al. "Wanna Make Your TCP Scheme Great for Cellular Networks? Let Machines Do It for You!", IEEE JSAC 2021 https://ieeexplore.ieee.org/document/9252929

Please note that if you're using v2.0-jsac-19 branch, you need to switch back to the kernel named c2tcp-v2.01, becasue compared to v2.0-jsac-19 kernel, some socket option name/features are changed in the new kernel.

Patching C2TCP Kernel: Compile the Kernel source code.

Another option is to compile your own kernel using the provided patch. You can use the instructions provided here to do that: https://github.com/Soheil-ab/C2TCP-IFIP/

The source code is available in linux-patch folder.

Verify the new kernel

uname -r

You should see something like 4.13.1-0521. If not follow the instructions on https://github.com/Soheil-ab/C2TCP-IFIP and bring the 4.13.1-0521 image on top of the grub list. For instance, you can use grub-customizer application.

Check whether C2TCP is there:

sysctl net.ipv4.tcp_c2tcp_enable

You should see:

net.ipv4.tcp_c2tcp_enable = 0

We will later enable C2TCP during our evaluation (In this version, in contrast with v2.0-jsac-19 branch, you can enable/disable C2TCP per socket. Still you can use net.ipv4.tcp_c2tcp_enable to enable C2TCP for the entire system though.)

In the current implementation, C2TCP's Tuner has been implemented in user-space. So, lets build C2TCP's Server-client app which includes The Tuner.

Build C2TCP's Server-Client App

Simply run the following:

cd ~/c2tcp/
./build.sh

Install our Cellular Traces gathered in NYC

You need to get the traces from following link and copy them in Mahimahi's folder

git clone https://github.com/Soheil-ab/Cellular-Traces-2018.git    
sudo cp Cell*/* traces/

Running The Evaluation

For the simplicity, first we disable password-timeout of sudo command:

sudo visudo

Now add following line and save it:

Defaults    timestamp_timeout=-1	

The Version includes standalone server-client applications (in contrast with v2.0-jsac-19 branch which had hardcoded mahimahi into the server.)
A minimal set of commands to run an evaluation and generate the results is put in run-sample.sh script. Simply run it as:

cd ~/c2tcp/
./run-sample.sh

If everything goes well, you should see a figure representing throughput through time. You can also find the summary of the results in log/summary.tr file.

No Target!

When you set target=0 (in run-sample.sh), system will switch to a mode where it sets the Target delay automatically to 2xminRTT. It is usefull in a more general scenario in which applcations might not have any specific delay target.

c2tcp's Issues

Compiling problems with c2tcp's linux debian file

Dear Soheil,
We have tried the new files you sent to us. The old errors have gone, though some new ones occur. Please see the following.

Thank you very much for the help.
Yongchang Hu

About multi-flow application issue

Dear Soheil,
We are trying to apply the C2TCP on multiple flows for testing the fairness. However, as we witness, when we use the idea of server-standalone.cc to control the flows, the C2TCP only successfully runs on a single flow.
Have you ever tested the multi-flows scenarios? Do you know why the aforementioned issue could happen?

Many thanks!
Best regards,
Yongchang

It's possible to apply the kernel patch to a newer kernel version?

When trying to apply linux-4-13-1-orca-c2tcp-0521.patch to the Debian 10 kernel version 4.19.0-16 I get:

patch: **** Only garbage was found in the patch input

Is it possible to apply the patch to a newer kernel version? Or I HAVE to revert to 4.13x if I want to try C2TCP?

Thanks.

No value for avg_urtt.

Dear Soheil,
We have tried the C2tcp. Though we keep getting avg_urtt = 0 at the followings.

c2tcp/src/server-standalone.cc

Line 264 in 7b84c91

u32 avg_ms=(deepcc_info.avg_urtt/1000);

Our plot shows:
avg_urtt:0 min_rtt:30814 cnt:0,avg_thr:0,thr_cnt:0,cwnd:18,pacing_rate:856164
min_rtt get non-zero value though.

BTW, why does C2tcp use deepcc struct, when there is tcp_c2tcp_info in define.h？

Yongchang

Regarding Client-Server App

Hi, I want to test c2tcp with my testbed of a real network environment, not with mahimahi trace data.
My server and client device has installed with your c2tcp kernel and I was able to compile all of your scripts.

Now I want to run client and server app with my real network.
Could you guide me on how to run your server and client app?

As I know, from your src>server.cc, I was able to find the usage
"./server [Delay(ms)] [port] [DL-trace] [UP-trace] [log] [Target] [Initial Alpha] [qsize in pkts] [Report Period: 1 sec] "
however, if I want to run this on two different device in real network, I don't know how to set up with DL-trace / UP-trace.

and client.cc,
"./client [server IP address] [flowid] [request data size(MB)] [dst_ip] [server port] "
what is difference between server IP address and dst_ip?

Thank you.

Error on c2tcp server and client app

Hello, with your kind guide, I built kernel 4.13.1 with your patch(linux-4-13-1-orca-c2tcp-0521.patch) and tried to run server and client app.

I set the server machine with this settings

sudo sysctl -w  net.ipv4.tcp_c2tcp_enable=1  # enable c2tcp
sudo sysctl -w net.ipv4.tcp_no_metrics_save=1 # disable tcp cache
sudo su
echo 0 > /sys/module/tcp_cubic/parameters/hystart  # disable tcp hystart
exit

sudo sysctl -w -q net.ipv4.tcp_low_latency=1
sudo sysctl -w -q net.ipv4.tcp_autocorking=0
sudo sysctl -w -q net.ipv4.tcp_no_metrics_save=1
sudo sysctl -w -q net.ipv4.ip_forward=0 # no using mahimahi

And, I've got this error.

$ sudo ./server 9001 0 150 100
[start_server] cubic
[start_server] Server is Connected to the client...
testing
[DataThread] Server is sending the traffic ...
[server-standalone.cc-CntThread-298] setsockopt: for index:0 flow_index:1 ... Protocol not available (ret1:-1)

Do I have to set the kernel of client machine same as the server?
I didn't install the kernel on client machine

Any further improvement about c2tcp?

hello, @Soheil-ab,
I have some questions to discuss with you,
First, what about the latest research progress about c2tcp?

Second, when apply c2tcp in cellular network, especially in 4G LTE scenario with highly bw and delay jitter, I think it is much more reasonable to adaptively update target delay according to the character of network, especially in wireless network. So is there any research about this, except for reinforcement learning based？

thx a lot,
Eric

Lack .ko files when doing modules_install in the compiling process

Sorry about interrupting again.
I tried to compile my own kernel following your guide in the c2tcp-IFID repo and I successfully compiled it using sudo make -j4. However, when I run sudo make modules_install, it warns me that it couldn't find xxx.ko files(which are all compiled into .o files). I was stuck in the step and don't really not how to solve this problem.

Have problem changing kernel to 4.13.1 in Ubuntu20.1

I am using Ubuntu20.1 on my PC and I have been trying to change the kernel for 2 days but still can't get it through.
I have followed the README sudo dpkg -i linux-image*521*,sudo dpkg -i linux-header*521*,sudo reboot and then verify the kernel but it was still 5.x. And then I followed your instructions in C2TCP-IFIP option2 step6.
After running sudo upgrade-from-grub-legacy, it gave me an error that dpkg-maintscript-helper: error: environment variable DPKG_MAINTSCRIPT_NAME is required. I have read some solution but I still couldn't understand how to solve it.
If I skip the problem above and run the following instructions and use grub-customizer to bring the 4.13.1 at the first line and its recovery mode to the second line and reboot. It still see 5.x when verifying the kernel. I really don't know how to solve it.

I am new to this process and I would be really grateful if you can help me with these problems.

Build kernel error

In file included from scripts/selinux/genheaders/genheaders.c:18:
./security/selinux/include/classmap.h:242:2: error: #error New address family defined, please update secclass_map.
  242 | #error New address family defined, please update secclass_map.
      |  ^~~~~
make[3]: *** [scripts/Makefile.host:101: scripts/selinux/genheaders/genheaders] Error 1
make[2]: *** [scripts/Makefile.build:561: scripts/selinux/genheaders] Error 2
make[1]: *** [scripts/Makefile.build:561: scripts/selinux] Error 2
make[1]: *** Waiting for unfinished jobs....
make: *** [Makefile:561: scripts] Error 2

How i use it in Ubuntu 19.04?

I am interested in optimizing the network. I want to try to optimize for international VPN lines.

How i use it in Ubuntu 19.04 (GNU/Linux 5.0.7-050007-lowlatency x86_64) ?