A little more than a month ago, the Open Compute Project celebrated its two-year anniversary. When Facebook first launched the project, it was our hope that we could spark more conversation and more collaboration around the development of efficient data center technologies. The OCP community has since grown to 50+ official members and thousands of participants; we’ve launched a foundation to guide the project; and we’ve started to advance open designs for everything from racks and storage boxes to motherboards and interconnects.

This is amazing progress in such a short span. But something’s missing. We are working together, in the open, to design and build smarter, more scalable, more efficient data center technologies — but we’re still connecting them to the outside world using black-box switches that haven’t been designed for deployment at scale and don’t allow consumers to modify or replace the software that runs on them.

With that in mind, we are today announcing a new project within OCP that will focus on developing a specification and a reference box for an open, OS-agnostic top-of-rack switch. Najam Ahmad, who runs the network engineering team at Facebook, has volunteered to lead the project, and a wide variety of organizations — including Big Switch Networks, Broadcom, Cumulus Networks, Facebook, Intel, Netronome, OpenDaylight, the Open Networking Foundation, and VMware — are already planning to participate. Work on the project will begin in earnest at the first-ever OCP Engineering Summit, being held at MIT on May 16.

It’s our hope that an open, disaggregated switch will enable a faster pace of innovation in the development of networking hardware; help software-defined networking continue to evolve and flourish; and ultimately provide consumers of these technologies with the freedom they need to build infrastructures that are flexible, scalable, and efficient across the entire stack. This is a new kind of undertaking for OCP — starting a project with just an idea and a clean sheet of paper, instead of building on an existing design that’s been contributed to the foundation — and we are excited to see how the project group delivers on our collective vision.

At the last Open Compute Summit, we talked about the importance of disaggregation, of separating the components of these technologies from each other so we can build systems that truly fit the workloads they run and update those components independently of each other, on an as-needed basis. But the promise of disaggregation — a promise that’s been made since the days of the mainframe — can truly be delivered on only if we work together, in the open, to establish common standards that everyone can adopt and build upon, from the bottom of the hardware stack to the top. And with the addition of this new project, the OCP community now has an opportunity to do exactly that.

We began sharing PUE for our Prineville data center at the end of Q2 2011 and released our first Prineville WUE in the summer of 2012. Now we’re pulling back the curtain to share some of the same information that our data center technicians view every day. We’ll continue updating our annualized averages as we have in the past, and you’ll be able to find them on the Prineville and Forest City dashboards, right below the real-time data.

Why are we doing this? Well, we’re proud of our data center efficiency, and we think it’s important to demystify data centers and share more about what our operations really look like. Through the Open Compute Project (OCP), we’ve shared the building and hardware designs for our data centers. These dashboards are the natural next step, since they answer the question, “What really happens when those servers are installed and the power’s turned on?”

Creating these dashboards wasn’t a straightforward task. Our data centers aren’t completed yet; we’re still in the process of building out suites and finalizing the parameters for our building managements systems. All our data centers are literally still construction sites, with new data halls coming online at different points throughout the year. Since we’ve created dashboards that visualize an environment with so many shifting variables, you’ll probably see some weird numbers from time to time. That’s OK. These dashboards are about surfacing raw data – and sometimes, raw data looks messy. But we believe in iteration, in getting projects out the door and improving them over time. So we welcome you behind the curtain, wonky numbers and all. As our data centers near completion and our load evens out, we expect these inevitable fluctuations to correspondingly decrease.

We’re excited about sharing this data, and we encourage others to do the same. Working together with AREA 17, the company that designed these visualizations, we’ve decided to open-source the front-end code for these dashboards so that any organization interested in sharing PUE, WUE, temperature, and humidity at its data center sites can use these dashboards to get started. Sometime in the coming weeks we’ll publish the code on the Open Compute Project’s GitHub repository. All you have to do is connect your own CSV files to get started. And in the spirit of all other technologies shared via OCP, we encourage you to poke through the code and make updates to it. Do you have an idea to make these visuals even more compelling? Great! We encourage you to treat this as a starting point and use these dashboards to make everyone’s ability to share this data even more interesting and robust.

Lyrica McTiernan is a program manager for Facebook’s sustainability team.

• Hired its first FTE (me)
• Initiated a new project (Compliance and Interoperability)
• Repurposed Virtual IO (Now Disaggregated IO)
• Signed up additional solution providers
• Held a staggeringly large January summit,
• Held its first international event in Hong Kong, and
• Launched the engineering summit program!

Unbelievable and only possible because of the great community that we’ve built together.  A wise person once said that open source projects live and die by their communities. If the past two years has been any indication, we are headed in the right direction!

As we continue to work together to grow our community and complete our mission of transforming the data center while truly democratizing the server, the Open Compute Project is both humbled and inspired by the many new paths that lay ahead.  Please join me in wishing the Open Compute Project a happy anniversary. Many thanks to all of you who work to make it one of the best open source projects around!

Cole Crawford
COO – Open Compute Foundation

Like in a software hackathon, the goal in a hardware hackathon is to take an idea from concept to prototype in as little time as possible. The big differences in a hardware hack are that the “prototype” is a set of manufacturing files (bill of materials, Gerbers, and solid models) and that projects are often judged not just on their innovation and utility, but also on how easily they could eventually be manufactured. At stake in this hack: The OCP Foundation would help the winning team patent their design or take it to prototype.

We began the process several weeks in advance, soliciting project ideas from the OCP community and zeroing in on six themes: storage, sensors & displays, communication, power, racks & mechanical, and robotics. My company, Upverter, provided design tools for the teams to use, and GrabCAD got involved to fill out the mechanical side of the hack.

And then came the main event! At 1:00 pm on the first day of the Summit the 70 hackers got together in one of the ballrooms. We walked through what tools were available and what type of files could be submitted at the end of the hack. We then started a timer and the hackers began to find and form teams.

The energy that poured into ideation and team formation was pretty awesome to watch. We had whiteboards set up all over the room, with little semicircles around each and hackers debating the problems they were excited about. After about 30 minutes teams were formed, ideas were picked, and the actual hacking began.

Here are the projects the different teams tackled:

19″ to Open Rack conversion

The challenge: Build mechanical and power interoperability between legacy compute hardware and the Open Rack standard. The hack was predominantly mechanical and resulted in a three-piece rack design that could be configured to accept either type of hardware. The team was working on the mechanical designs at the end of the hack.

Bluetooth adaptor

The challenge: Get rid of the USB and serial cable mess that’s required to hook a laptop up to the debug port of a server. The hack designed a small circuit that allowed the technician to connect to a server via Bluetooth, instead of via a serial cable. The team had a working prototype and design files in Upverter at the end of the hack.

Debug port aggregation

The challenge: Collect all the debug port information from all the servers in a rack and connect it to the Internet. The hack was designed mechanically and electrically and was mostly complete by the end of the hack.

The “FlexiStrip”

The challenge: Line a smattering of server racks from floor to ceiling with an array of temperature sensors and then use that data to generate a thermal profile of the data center. The team focused on the sensor arrays themselves, but they also did some Arduino prototyping into reading sensors and wireless communication.

The “Happy Rack”

The challenge: Monitor the health and happiness of a server rack and display it prominently and physically on the top of the rack. The hack was done as a prototype cardboard box with an Arduino-driven arrow that moved between an unhappy face and a happy face depending on input. (This hack won our unofficial “Hacker Spirit Award.”)

High-density Open Rack JBOD server

The challenge: Take a high-density array of cheap disks and design an enclosure for them that would fit into an OCP Open Rack. The team made it as far as mechanical and thermal models. They were beginning to work on the carrier/backplane circuit boards at the end of the hack.

Mesh-networked sensors

The challenge: Distribute a large number of low-power sensors around a data center and communicate between them efficiently. The hack was built on top of Arduino, ZigBee wireless, and a simple thermal sensor. The team focused on a prototype, finished it early, and spent the rest of the hack on documentation and moving the design into Upverter.

And the winner is…

At the end of the hackathon, all the groups did quick presentations on their hacks. We were all impressed by the quality, applicability, and completeness of all of the projects. To determine the winner, we looked at the five deliverables (schematic, layout, mechanical, prototype, documentation), the hack’s applicability to data centers, and the hack’s completeness and manufacturability.

The winning design: The debug port aggregation hack. This team produced a design that hit 4 of the 5 deliverables, was super applicable, and was just hours from completion.

Congratulations to all the teams involved for their great work, and see you at the next OCP hardware hack!

Zak Homuth is the co-founder and CEO of Upverter. Upverter builds the pickaxes of hardware engineering. Ideation tools, schematic capture, PCB layout, design hosting, parts library management, and affordable prototyping. All of which are collaborative, accessible, reusable, and in the cloud.

I posted a request with the drawings they had and asked for help. Within a few hours one of our community members let me know that 3M had all the CAD files they needed, deeply buried on their site!

The models were not in our library directly, which isn’t a problem because it’s the power of our community that gets the job done. Our resources are growing each day and the goal is to make engineers’ lives better with stories just like this.

Sara Sigel is the community manager at GrabCAD.

In October 2011 we announced the formation of a non-profit foundation to steer the project, and that organization has continued to evolve. In the last six months we’ve hired our first full-time employee, COO Cole Crawford, and added a new Compliance & Interoperability project. We’ve also seen our first international expansion, with the launch of an OCP Asia-Pacific chapter in Tokyo, and some encouraging engagement with the academic and research communities — including a Purdue University competition to design a biodegradable server chassis.

We’ve also seen some tremendous engagement in the broader OCP community in the last six months. Some examples include:

• New members: More than a dozen new organizations have joined, including storage companies like EMC, Fusion-io, Hitachi, and Sandisk; microprocessor organizations like Applied Micro, ARM, Calxeda, and Tilera; and communications companies like NTT Data and Orange.
• Adoption of open source hardware: More companies are being inspired to think differently about how they build out their infrastructures. Examples include Riot Games, which has decided to purchase systems based on OCP designs and built by Hyve Solutions, and Rackspace, which has decided to design and build its own infrastructure using OCP designs as a starting point.
• Momentum around Open Vault and Open Rack: We launched these two projects at the third OCP Summit in May 2012, and we’re already seeing companies iterating on them in interesting ways and developing creative new technologies to complement them. Examples include new Open Rack designs from Avnet, Delta, Emerson, and Sanmina (many of which are being demonstrated at today’s Summit) and new server designs for the SAS expander slot on the Open Vault spec from Calxeda and Intel.
• Products being delivered: Also at the May 2012 Summit, AMD and Intel announced that they’d been collaborating with Fidelity and Goldman Sachs to develop new boards to suit those companies’ specific workloads — codenamed Roadrunner and Decathlete, respectively. Just seven months later, both companies are here today demonstrating finished products.
• New technology contributions: We’re starting to see significant momentum building behind technology contributions to the project. Some of the new contributions include designs for Fusion-io’s new 3.2TB ioScale card; the designs Rackspace has created for its new infrastructure; and designs for a new storage box that Hyve Solutions built for Open Rack. Facebook is also making a few new contributions, including modifications to Open Rack and Open Vault for use in cold-storage environments and designs for a new, all-flash database server (codenamed Dragonstone) and the latest version of its web server (codenamed Winterfell).

But most exciting of all are a series of new developments that will enable us to take some big steps forward toward better utilization of these technologies. One of the challenges we face as an industry is that much of the hardware we build and consume is highly monolithic — our processors are inextricably linked to our motherboards, which are in turn linked to specific networking technology, and so on. This leads to poorly configured systems that can’t keep up with rapidly evolving software and waste lots of energy and material.

To fix this, we need to break up some of these monolithic designs — to disaggregate some of the components of these technologies from each other so we can build systems that truly fit the workloads they run and whose components can be replaced or updated independently of each other. Several members of the Open Compute Project have come together today to take the first steps toward this kind of disaggregation:

• Silicon photonics: Intel is contributing designs for its forthcoming silicon photonics technology, which will enable 100 Gbps interconnects — enough bandwidth to serve multiple processor generations. This technology also has such low latency that we can take components that previously needed to be bound to the same motherboard and begin to spread them out within a rack.
• “Group Hug” board: Facebook is contributing a new common slot architecture specification for motherboards. This specification — which we’ve nicknamed “Group Hug” — can be used to produce boards that are completely vendor-neutral and will last through multiple processor generations. The specification uses a simple PCIe x8 connector to link the SOCs to the board.
• New SOCs: AMD, Applied Micro, Calxeda, and Intel have all announced support for the Group Hug board, and Applied Micro and Intel have already built mechanical demos of their new designs.

Taken together, these announcements will enable data center operators to build systems that better fit the workloads they need to run and to upgrade through multiple generations of processors without having to replace the motherboards or the in-rack networking. This should in turn enable real gains in utilization and unlock even more efficiencies in data center construction and operations.

There’s a lot of work left to do here, but it’s never been more important for us to take these steps. As our lives become more connected, and as more devices and applications generate more data, we will face compute and storage challenges that existing technologies cannot handle efficiently.

But we have one big advantage, as we face these challenges: We are doing this work together, in the open, and everyone has a chance to contribute — to help ensure that all the technologies we develop and consume are as scalable as possible, as efficient as possible, and as innovative as possible.

You can see the list of keynote speakers and read the rundown of executive sessions and technical tracks on the summit’s agenda page.

See you next week in Santa Clara, California!

The C&I project is the seventh official OCP working group, and it will focus on establishing a transparent and scalable framework to ensure that solutions meet OCP standards for compliance and interoperability and are branded accordingly. This project will allow providers to leverage a common set of tools and processes for delivering solutions that meet OCP-defined expectations. The more thoughtful and robust the approach, the better for the OCP community.

And this is where you come in.

We invite you to join us at the fourth Open Compute Summit on Jan. 16-17 in Santa Clara, where we will host a Compliance and Interoperability technical track focused on discussing he most effective ways to certify OCP solutions. There is a vast amount of collective certification expertise within the OCP community, and the Summit offers a wonderful opportunity to begin harnessing that expertise for the greater good. We will also be looking for leaders to help us drive forward projects that will emerge out of the C&I project.

We look forward to seeing you at the OCP Summit. In the meantime, feel free to share any questions or comments by contacting us at opencompute-ci@lists.opencompute.org.

Eric is Vice President of Data Center Services at Fidelity Investments.

• We’re finalizing the agenda now and will have a great lineup of keynote speakers, executive sessions, and technical tracks. Tim O’Reilly will deliver one of the keynote addresses; Tim brings a deep commitment to open source and we’re looking forward to the insights he’ll share.
• We’re hosting our first-ever hardware hackathon during the summit. Register now to hack with fellow attendees and help drive change across the industry.
• If you plan to attend one of the technical tracks at the summit — Open Rack; Open Vault; open hardware management; virtual IO; motherboards; or OCP compliance and interoperability — please fill out this one-question survey to let us know which track you’re interested in.  Keep in mind that the technical tracks run at the same time as the executive sessions and the hackathon.