slidesSurge is brought to you by OmniTI, the leading Web Scalability and Performance provider.

speakers

Mark Cavage

Software Engineer, Joyent

@mcavage

David Pacheco

Software Engineer, Joyent

@dapsays

  • Track: Scaling Architecture
    Scaling the Unix Philosophy to Big Data

    Manta is a system we have developed that applies the Unix philosophy—small, well-defined tools and simple ways to combine them—to distributed computing on a multi-tenant object store. This new model of general-purpose compute unifies such disparate big data applications as indexing, log analysis, image processing and video transcoding, but also carries with it a new set of design and implementation challenges.

    In this talk, we'll discuss these challenges and our experiences developing, deploying, debugging and scaling this system. We will pay particular attention to both the design choices validated by implementation and those that needed to be revisited—or entirely rethought—to maintain low latency as we scaled the system. Finally, in classic Surge fashion, we will discuss those system-wide failures that resulted in prolonged outages, but ultimately yielded a much more robust system.

Mark Cavage

Mark Cavage is a Software Engineer at Joyent, where he works primarily on distributed systems supporting Joyent's cloud computing suite, and maintains several popular open source projects, such as restify and ldapjs. Prior to joining Joyent, Mark was a Senior Software Engineer with Amazon Web Services, where he was primarily responsible for envisioning and launching the AWS Identity and Access Management product.

David Pacheco

David Pacheco is a software engineer at Joyent, where he led the development of Joyent's Cloud Analytics system for observing software components distributed across a datacenter. More recently, he has worked on tools for debugging Node.js programs in production, both live and postmortem.

about

Other Meaningful Information