Brendan Gregg Lead Performance Engineer, Joyent
Brendan Gregg is the lead performance engineer at Joyent, where he analyzes performance and scalability for small to large cloud computing environments, at any level of the software stack, down to metal. He is the primary author of "DTrace" (Prentice Hall, 2011), and co-author of "Solaris Performance and Tools" (Prentice Hall, 2006), and well as numerous articles about systems performance. He was previously a performance lead and kernel engineer at Sun Microsystems, and has also been a performance consultant. He developed the DTraceToolkit and the ZFS L2ARC, and many of his DTrace scripts are shipped by default in Mac OS X and Oracle Solaris 11. His recent work has included performance visualizations for illumos and Linux kernel analysis.
The Real-Time Web in the Real World: DIRT in Production
At the Surge 2010 keynote, we described an emerging breed of web-facing applications that exhibited data-intensive real-time semantics (semantics we dubbed "DIRT"). Now, two years later, the first generation of these DIRTy applications have been deployed into wide-scale production, and we have experienced first-hand the new challenge that they present: the explosiveness of demand of the social web but with the constraints of a real-time system—and all via brittle, failure-prone mobile networks. In this talk, we will describe our experiences as the server-side infrastructure provider for one such application, Voxer—a popular push-to-talk mobile app. We will describe how we debugged latency-inducing issues at all layers of the stack: from the bowels of the operating system (I/O, networking, CPU scheduling) through platform (node.js, Erlang/Riak) and into the application. We will describe some of the new challenges that are posed by asynchronous, event-oriented systems like node.js — especially when deployed on mobile applications. Finally, we will describe some of the debugging and visualization technology that we have developed to help us better understand and diagnose these systems—and demonstrate their efficacy on actual production problems.