jm + udp   17

Internet protocols are changing
per @mnot. HTTP/2; TLS 1.3; QUIC and UDP; and DOH (DNS over HTTP!)
crypto  encryption  http  https  protocols  http2  tls  quic  udp  tcp  dns  tunnelling 
5 days ago by jm
NVD - CVE-2016-10229
udp.c in the Linux kernel before 4.5 allows remote attackers to execute arbitrary code via UDP traffic that triggers an unsafe second checksum calculation during execution of a recv system call with the MSG_PEEK flag.
udp  security  cve  linux  msg_peek  exploits 
april 2017 by jm
About to leave UPC due to (lack of) port forwarding - Boards.ie
Virgin Media/UPC seem to have silently deployed an IPv6 "carrier-grade NAT" setup called "DS-Lite" -- ie. all customers now get just a routable IPv6 address, and share a small pool of IPv4 NATs. This breaks a multitude of useful services, including UDP IPSec VPNs it seems
udp  vpns  isps  virgin-media  virgin  ireland  ds-lite  ipv6  tunnelling  networking  nat  ipv4 
may 2016 by jm
'Devastating' bug pops secure doors at airports, hospitals
"A command injection vulnerability exists in this function due to a lack of any sanitisation on the user-supplied input that is fed to the system() call," Lawshae says.


:facepalm:
security  iot  funny  fail  linux  unix  backticks  system  udp  hid  vertx  edge 
april 2016 by jm
Topics in High-Performance Messaging
'We have worked together in the field of high-performance messaging for many years, and in that time, have seen some messaging systems that worked well and some that didn't. Successful deployment of a messaging system requires background information that is not easily available; most of what we know, we had to learn in the school of hard knocks. To save others a knock or two, we have collected here the essential background information and commentary on some of the issues involved in successful deployments. This information is organized as a series of topics around which there seems to be confusion or uncertainty. Please contact us if you have questions or comments.'
messaging  scalability  scaling  performance  udp  tcp  protocols  multicast  latency 
december 2015 by jm
How to receive a million packets per second on Linux

To sum up, if you want a perfect performance you need to:
Ensure traffic is distributed evenly across many RX queues and SO_REUSEPORT processes. In practice, the load usually is well distributed as long as there are a large number of connections (or flows).
You need to have enough spare CPU capacity to actually pick up the packets from the kernel.
To make the things harder, both RX queues and receiver processes should be on a single NUMA node.
linux  networking  performance  cloudflare  packets  numa  so_reuseport  sockets  udp 
june 2015 by jm
David P. Reed on the history of UDP
'UDP was actually “designed” in 30 minutes on a blackboard when we decided pull the original TCP protocol apart into TCP and IP, and created UDP on top of IP as an alternative for multiplexing and demultiplexing IP datagrams inside a host among the various host processes or tasks. But it was a placeholder that enabled all the non-virtual-circuit protocols since then to be invented, including encapsulation, RTP, DNS, …, without having to negotiate for permission either to define a new protocol or to extend TCP by adding “features”.'
udp  ip  tcp  networking  internet  dpr  history  protocols 
april 2015 by jm
Moving Big Data into the Cloud with Tsunami UDP - AWS Big Data Blog
Pretty serious speedup. 81 MB/sec with Tsunami UDP, compared to 9 MB/sec with plain old scp. Probably kills internet performance for everyone else though!
tsunami-udp  udp  scp  copying  transfers  internet  long-distance  performance  speed 
august 2014 by jm
Tracedump
a single application IP packet sniffer that captures all TCP and UDP packets of a single Linux process. It consists of the following elements:

* ptrace monitor - tracks bind(), connect() and sendto() syscalls and extracts local port numbers that the traced application uses;
* pcap sniffer - using information from the previous module, it captures IP packets on an AF_PACKET socket (with an appropriate BPF filter attached);
* garbage collector - periodically reads /proc/net/{tcp,udp} files in order to detect the sockets that the application no longer uses.

As the output, tracedump generates a PCAP file with SLL-encapsulated IP packets - readable by eg. Wireshark. This file can be later used for detailed analysis of the networking operations made by the application. For instance, it might be useful for IP traffic classification systems.
debugging  networking  linux  strace  ptrace  tracedump  tracing  tcp  udp  sniffer  ip  tcpdump 
may 2014 by jm
Game servers: UDP vs TCP
this HN thread on the age-old UDP vs TCP question is way better than the original post -- lots of salient comments
udp  tcp  games  protocols  networking  latency  internet  gaming  hackernews 
april 2014 by jm
DNS results now being manipulated in Turkey
Deep-packet inspection and rewriting on DNS packets for Google and OpenDNS servers. VPNs and DNSSEC up next!
turkey  twitter  dpi  dns  opendns  google  networking  filtering  surveillance  proxying  packets  udp 
march 2014 by jm
An Empirical Evaluation of TCP Performance in Online Games
In this paper, we have analyzed the performance of TCP in of ShenZhou Online, a commercial, mid-sized MMORPG. Our study indicates that, though TCP is full-fledged and robust, simply transmitting game data over TCP could cause unexpected performance problems. This is due to the following distinctive characteristics of game traffic: 1) tiny packets, 2) low packet rate, 3) application-limited traffic generation, and 4) bi-directional traffic.

We have shown that because TCP was originally designed for unidirectional and network-limited bulk data transfers, it cannot adapt well to MMORPG traffic. In particular, the window-based congestion control mechanism and the fast retransmit algorithm for loss recovery are ineffective. This suggests that the selective acknowledgement option should be enabled whenever TCP is used, as it significantly enhances the loss recovery process. Furthermore, TCP is overkill, as not every game packet needs to be transmitted reliably and processed in an orderly manner. We have also shown that the degraded network performance did impact users' willingness to continue a game. Finally, a number of design guidelines have been proposed by exploiting the unique characteristics of game traffic.


via Nelson
tcp  games  udp  protocols  networking  internet  mmos  retransmit  mmorpgs 
november 2013 by jm
"High Performance Browser Networking", by Ilya Grigorik, read online for free
Wow, this looks excellent. A must-read for people working on systems with high-volume, low-latency phone-to-server communications -- and free!
How prepared are you to build fast and efficient web applications? This eloquent book provides what every web developer should know about the network, from fundamental limitations that affect performance to major innovations for building even more powerful browser applications—including HTTP 2.0 and XHR improvements, Server-Sent Events (SSE), WebSocket, and WebRTC.

Author Ilya Grigorik, a web performance engineer at Google, demonstrates performance optimization best practices for TCP, UDP, and TLS protocols, and explains unique wireless and mobile network optimization requirements. You’ll then dive into performance characteristics of technologies such as HTTP 2.0, client-side network scripting with XHR, real-time streaming with SSE and WebSocket, and P2P communication with WebRTC.

Deliver optimal TCP, UDP, and TLS performance;
Optimize network performance over 3G/4G mobile networks;
Develop fast and energy-efficient mobile applications;
Address bottlenecks in HTTP 1.x and other browser protocols;
Plan for and deliver the best HTTP 2.0 performance;
Enable efficient real-time streaming in the browser;
Create efficient peer-to-peer videoconferencing and low-latency applications with real-time WebRTC transports


Via Eoin Brazil.
book  browser  networking  performance  phones  mobile  3g  4g  hsdpa  http  udp  tls  ssl  latency  webrtc  websockets  ebooks  via:eoin-brazil  google  http2  sse  xhr  ilya-grigorik 
october 2013 by jm
packetdrill - network stack testing tool
[Google's] packetdrill scripting tool enables quick, precise tests for entire TCP/UDP/IPv4/IPv6 network stacks, from the system call layer down to the NIC hardware. packetdrill currently works on Linux, FreeBSD, OpenBSD, and NetBSD. It can test network stack behavior over physical NICs on a LAN, or on a single machine using a tun virtual network device.
testing  networking  tun  google  linux  papers  tcp  ip  udp  freebsd  openbsd  netbsd 
july 2013 by jm
Building a Modern Website for Scale (QCon NY 2013) [slides]
some great scalability ideas from LinkedIn. Particularly interesting are the best practices suggested for scaling web services:

1. store client-call timeouts and SLAs in Zookeeper for each REST endpoint;
2. isolate backend calls using async/threadpools;
3. cancel work on failures;
4. avoid sending requests to GC'ing hosts;
5. rate limits on the server.

#4 is particularly cool. They do this using a "GC scout" request before every "real" request; a cheap TCP request to a dedicated "scout" Netty port, which replies near-instantly. If it comes back with a 1-packet response within 1 millisecond, send the real request, else fail over immediately to the next host in the failover set.

There's still a potential race condition where the "GC scout" can be achieved quickly, then a GC starts just before the "real" request is issued. But the incidence of GC-blocking-request is probably massively reduced.

It also helps against packet loss on the rack or server host, since packet loss will cause the drop of one of the TCP packets, and the TCP retransmit timeout will certainly be higher than 1ms, causing the deadline to be missed. (UDP would probably work just as well, for this reason.) However, in the case of packet loss in the client's network vicinity, it will be vital to still attempt to send the request to the final host in the failover set regardless of a GC-scout failure, otherwise all requests may be skipped.

The GC-scout system also helps balance request load off heavily-loaded hosts, or hosts with poor performance for other reasons; they'll fail to achieve their 1 msec deadline and the request will be shunted off elsewhere.

For service APIs with real low-latency requirements, this is a great idea.
gc-scout  gc  java  scaling  scalability  linkedin  qcon  async  threadpools  rest  slas  timeouts  networking  distcomp  netty  tcp  udp  failover  fault-tolerance  packet-loss 
june 2013 by jm
pwnat - NAT to NAT client-server communication
'a proxy server that works behind a NAT, even when the client is behind a NAT, without any 3rd party'. nifty, by Samy "MySpace worm" Kamkar
samy-kamkar  apps  firewall  ip  nat  networking  pwnat  stun  traversal  tcp  sysadmin  tunneling  udp  from delicious
march 2010 by jm

related tags

3g  4g  alerting  apps  async  backticks  book  browser  canaries  cloudflare  copying  crypto  cve  debugging  distcomp  dns  dpi  dpr  ds-lite  ebooks  edge  encryption  exploits  fail  failover  fault-tolerance  filtering  firewall  freebsd  funny  games  gaming  gc  gc-scout  google  graphite  hackernews  hid  history  hostedgraphite  hsdpa  http  http2  https  ilya-grigorik  internet  iot  ip  ipv4  ipv6  ireland  isps  java  latency  linkedin  linux  long-distance  messaging  mmorpgs  mmos  mobile  monitoring  msg_peek  multicast  nat  netbsd  netty  networking  numa  openbsd  opendns  ops  packet-loss  packets  papers  performance  phones  protocols  proxying  ptrace  pwnat  qcon  quic  rest  retransmit  samy-kamkar  scalability  scaling  scp  security  slas  sniffer  sockets  so_reuseport  speed  sse  ssl  strace  stun  surveillance  sysadmin  system  tcp  tcpdump  testing  threadpools  timeouts  tls  tracedump  tracing  transfers  traversal  tsunami-udp  tun  tunneling  tunnelling  turkey  twitter  udp  unix  vertx  via:eoin-brazil  virgin  virgin-media  vpns  webrtc  websockets  xhr 

Copy this bookmark:



description:


tags: