Open Compute Project

Just six months ago, we announced our intention to expand the charter of the Open Compute Project to include networking hardware.

At the time, it was our hope that we could build on the momentum we'd established in opening up server, storage, and datacenter designs and collaborate with the broader community on the development of an open, OS-agnostic top-of-rack switch. Such a switch, we believed, would enable a faster pace of innovation in the development of networking hardware; help software-defined networking continue to evolve and flourish; and ultimately provide consumers of these technologies with the freedom they need to build infrastructures that are flexible, scalable, and efficient across the entire stack.

Our progress so far has exceeded even our lofty expectations -- hundreds of people are actively collaborating on the development of more than 30 potential contributions, covering most of the network hardware stack and even some of the network software stack.

We'd like to highlight four contributions in particular, all of them currently being considered by the OCP Incubation Committee and likely to be accepted soon.

Broadcom was the first to develop a full specification for and implementation of an open switch. Here's how they describe it:

"Broadcom has developed an Open Network Switch specification, addressing popular leaf and spine switch configurations and feature requirements, in compliance with the charter defined by the OCP networking initiative. The specification delivers the foundation for efficient, high performance, and flexible network architectures, complementing the goals of the OCP networking initiatives. Our network switch specification is based on the widely deployed Trident switch architecture, which supports a wide ecosystem of networking operating systems and applications. The specification utilizes the latest in the Trident family, the Trident II, bringing the most advanced and comprehensive feature set into the open switch ecosystem. We have been successful in delivering the first version of the specification and working switch systems from our hardware partner that complies with the specification -- all in less than six months. We believe this specification will enable faster innovation in the market and more choice for data center operators and telecom service providers. " 

Cumulus Networks has proposed its Open Network Install Environment (ONIE) software as a contribution to OCP:

"ONIE, which was introduced by Cumulus Networks and is supported by networking OEM, ODM, and communication silicon vendors across the globe, is an industry standard network boot loader to install software on network switches, thus enabling a bare metal Ethernet switch ecosystem. ONIE defines a runtime install environment that supports multiple network operating system vendors at scale that -- for the first time -- effectively provides customers more control and the ability to choose when it comes to their networking hardware and software. ONIE's open install environment can be supported on a range of existing ODM switches, as well as the open network switch design specifications being developed by the Open Compute Project, ultimately enabling end users to select among different network operating systems and a variety of compatible hardware."

Intel has also developed a specification for an open switch, and they describe it as follows:

"Intel’s proposed contribution to the Open Compute Project network working group is a specification for a bare-metal, top-of-rack switch. The specification describes a 48x4 10/40G switch including all necessary subsystems for switching, control CPU, peripherals, external interfaces, power, cooling, and mechanical enclosure. An example of a switch that adheres to this specification, based on Intel parts, can be found here. Platforms based on this spec enable more choice, improved flexibility, and a better cost structure for customers who choose to implement a software defined approach for networking and switching. To complement the proposed contribution to the OCP working group, Intel brings an ecosystem of partners ready to supply production level systems with a variety of solution capabilities."

Finally, Mellanox is the third company to have developed a specification for an open switch. Here's their description:

"Mellanox is expanding its Open Compute Project portfolio offering with the proposed contribution of its SwitchX-2 x86-based top-of-rack switch specification. The switch supports 48 SFP+ ports and 12 QSFP ports, enabling non-blocking connectivity within the OCP Open Rack, or alternatively, enabling 60 10GbE server ports when using QSFP+ to SFP+ breakout cables to increase rack efficiency for less bandwidth demanding applications. The new switch will be the first to enable ONIE over x86, and we expect it to dramatically improve power consumption, latency, and density and enable larger, more efficient, and more cost-effective datacenter designs."

Taken together, these contributions are tremendous steps forward toward our vision of a truly disaggregated network stack. They are also vivid proof of the OCP community's ability to work together, in the open, to develop innovative new technologies -- and to do so at an almost unheard-of pace.

We will continue our work on these technologies and others later this week, at the OCP engineering workshop being held at University of Texas at San Antonio. We hope to see you there!

Continue reading this post

It's been a year since we last came together, and our momentum continues to build. Our community now numbers in the thousands -- more than 3,400 people registered for the fifth Open Compute Summit, which is now in progress in San Jose -- and we have more than 150 official members, including new partners like Bloomberg, Box, Cumulus Networks, IBM, IO, LSI, Microsoft, and Yandex. We are also growing rapidly outside North America, with OCP Japan and OCP Taiwan bringing together dozens of new members in its first year as a chapter, and new chapters beginning to form in Europe, Korea, Philippines, and Australia / New Zealand. 

We've accomplished a lot in the last year: We expanded the foundation staff, welcoming Amber Graner as community manager and Hugh Blemings as director of certification. We held engineering workshops every six weeks, bringing together thousands of participants from around the world to advance the work of each of our projects. And we launched a new project in 2013: networking. Our aim with this new project was to begin to open up the network hardware stack with the development of an open, OS-agnostic switch. This was a new challenge for us -- typically our projects have been formed around existing contributions -- but we delivered, and we did it quickly: After just six months, we unveiled the first set of contributions in the project, from Broadcom, Cumulus Networks, Intel, and Mellanox.

The pace of innovation in the industry has also continued to accelerate, and in the past year we've seen a wealth of OCP-inspired technology advancements and contributions from across the community. Here are just a few examples: 

  • AMD: Today at the summit, AMD showcased a development platform for its first 64-bit ARM-based server CPU and contributed a new microserver design to OCP that is compatible with the common-slot architecture specification dubbed “Group Hug.”

  • Facebook: Facebook has contributed its new "Honey Badger" microserver adapter to OCP, and today at the summit they showcased their new rapid deployment datacenter concepts and their new optical storage prototype. Facebook also shared that OCP and related efficiency efforts have helped the company save more than $1.2 billion in infrastructure costs over the last three years.

  • Fidelity: Fidelity has contributed the designs for its "Open Bridge Rack," which enables the deployment of OCP storage and server designs in legacy 19" racks. 

  • Hyve: Hyve has contributed the designs for its 1500 Series server designs, which are also designed to fit into legacy 19" racks.

  • IO: Also today, IO announced a new containerized data center solution that employs OpenStack and Open Compute Project hardware. IO already has several customers for this new service, including Merck.

  • Microsoft: Yesterday, Microsoft announced that it has joined the Open Compute Project and is contributing designs for the servers that power global cloud services like Windows Azure, Office 365, and Bing.   

  • LSI: LSI announced that it has joined the Open Compute Project, and they immediately contributed two designs: a 12G SAS expander upgrade to the Open Vault storage system and a flash storage card that provides low-latency flash storage to server-based applications. 

  • Seagate: Seagate has contributed its Kinetic open storage platform, which is designed to prevent scale bottlenecks in storage.

  • Quanta: Quanta has contributed the entire line of Open Rack-compatible products they co-developed with Rackspace. 

As impressive as all of this is, some of the work I'm most proud of from the past year has been in our efforts to make it easier for people to collaborate on the development of new OCP technologies, contribute those technologies to the OCP Foundation, and consume them in whatever combination best fits their needs:

  • Supporting the OCP Solution Provider ecosystem: We've seen a lot of growth here in the last year, and there are now seven official OCP Solution Providers: AMAX, Avnet, CTC, Hyve, Penguin Computing, Quanta, and Racklive. These companies provide new options for consumers who want to deploy OCP designs, and new routes to market for innovative new technologies. All these companies are making big investments in OCP, building labs of their own and contributing the designs for the OCP products they develop for customers back to the foundation.

  • New certification process: We've developed a rigorous compliance and interoperability process, with two levels of certification: "OCP Ready" and "OCP Certified." These certifications will provide consumers with assurance that the products with these labels have been thoroughly tested and meet the standards set by the OCP community. Two new labs have been established -- UTSA and ITRI -- to manage the certification process. Wiwynn was the first company to successfully achieve OCP certification for one of their products, Quanta quickly followed. 

  • New OCP hardware license: Since our inception a little more than two years ago, we've used a relatively "permissive" license (modeled on Apache) to govern contributions. Soon we will roll out a second, more "prescriptive" license (modeled on GPL) that will require anyone who modifies an original design and then sells that design to contribute the modified version back to the foundation. It's our hope that having multiple licensing options will lead to even more OCP technology contributions.

Looking at all this progress and forward momentum, I can't help but think that 2014 is our year. New technologies are being developed and contributed; new products and new businesses are being launched; and new OCP technologies are being adopted. We are reinventing this industry together, in the open, and everyone has a chance to contribute — to help ensure that all the technologies we develop and consume are as scalable as possible, as efficient as possible, and as innovative as possible.

Thanks to all of your hard work, the future is open. So let's keep going!

 

 

Continue reading this post

I have some exciting news to share: The Open Compute Project Foundation board has voted to expand to include two new members. Bill Laing, Corporate Vice President of Cloud and Enterprise at Microsoft, and Jason Taylor, Director of Infrastructure at Facebook, have joined the board effective immediately. I recently left Facebook to pursue a new OCP-related startup opportunity, and will remain on the board as an independent. The board has also voted to retain me as the president and chairman of the foundation.

The Open Compute Project continues to gain momentum, and Bill and Jason will be great additions to the community's leadership. Together we will continue to accelerate the pace of innovation in this industry and to make datacenter technologies more open, more efficient, and more sustainable.Expanding the OCP Foundation Board

Continue reading this post

Open Compute Delivers Flexibility and Empowerment to Fidelity Investments
Fidelity Investments reconfigured its data centers to better fit its business needs, engaging its engineering team in redesigning a revolutionary new rack, and reducing energy consumption by 20%.

AT A GLANCE

Industry: Financial Services

Data centers:  4

Physical servers: 11,000

Virtual servers: 22,000

Storage: 22PB

IT staff: 10,000+

Founded in 1946, Fidelity Investments is a multinational financial services corporation headquartered in Boston, Massachusetts. It is one of the largest mutual fund and financial services groups in the world.

Fidelity’s Enterprise Infrastructure team is responsible for the company’s data centers, networks, technology platforms and enterprise software, providing the underlying technology that enables the company’s IT developers to deliver timely, innovative solutions and services to its customers.

CHALLENGE
Finding new efficiencies
In the wake of the 2008 financial crisis, Fidelity Investments was looking for opportunities to streamline its operations. The company’s Enterprise Infrastructure team found room for improvement in the way it consumed technology in its servers and data centers.

“Many enterprises have been sold on proprietary gear for years,” says Eric Wells, vice president of data center services for Fidelity Investments. “At Fidelity, we’d gotten used to our suppliers telling us what was available, what the next innovations were. That’s fine, but it drives a very high-cost operating model. We needed to do it differently going forward.”

Fidelity’s Enterprise Infrastructure team also wanted to transform that way its members worked. Instead of maintaining a closed shop, the team was looking to open up and engage with an external community of engineers as a way to keep up with the latest developments and innovate in-house.

The Enterprise Infrastructure team is focused on three key things: time-to-market, cost efficiency and system security/stability. It was looking to build an underlying architecture that supported all of these things, and would also help reduce operational cost and complexity.

SOLUTION
Reinventing the data center
Open Compute is an open-source community of engineers working to redesign servers, storage and data center hardware designs for maximum efficiency and scalability. Open Compute was born inside Facebook, when the company realized that it had to re-invent the data center to maintain its exponential growth rate. Facebook designed and built a new kind of data center from the ground up that is 38% more efficient and 24% less expensive to run than a traditional data center. The company then shared these technologies as the Open Compute Project in 2011, hoping to create an open-source movement in the data center space that would bring about the same kind of rapid innovation typical of open source software.

Today, OCP has 150 member companies and thousands of participants working on eight different projects.

For Fidelity, Open Compute came along at just the right time. Since joining Open Compute in 2011, the company’s hardware engineers have participated in a number of initiatives, held a variety of positions and contributed several key specs back to the community. “We’re excited and proud to be part of a community that complements our own innovation in data center design, cloud services, social media and mobile devices,” says Keith Shinn, senior vice president, enterprise infrastructure. The company is currently migrating much of its distributed IT footprint to a private cloud platform based on Open Compute—a project that wasn’t possible before Fidelity designed the Open Bridge Rack.

Open Bridge Rack

The Open Bridge Rack is a convertible datacenter rack that can physically adjust to hold any size server up to 23 inches across. It’s a key link in helping enterprises make the switch from proprietary gear to Open Compute.

Traditionally, servers and racks are sold together. Standard servers are 19 inches wide while OCP servers (21 inches) and Telco devices (23 inches) are wider, so they require a larger rack. But even within 19-inch servers, there are slight size differences between different vendors, making their servers and racks incompatible. This is one way suppliers ensure future business.

The Open Bridge Rack changes all that, because it can be adjusted to accommodate a 19-inch server today, and a 21 or 23-inch server tomorrow, allowing Fidelity—or any enterprise—to future-proof its investment in racks.

The rack also improves on the standard power supply configuration. Known to be a big mark-up item for vendors, standard racks have 80 power supplies for 40 servers—many more than strictly necessary. What’s more, they can’t be replaced if they fail. The result is energy inefficiency, additional cost and wasted material. The Open Bridge Racks, like their predecessor the Open Compute Open Rack, have just 12 power supplies for 40 servers. They’re easy to replace if they fail, and they can be reused when an enterprise refreshes its compute.

Not only does Fidelity save money by not having to buy new racks, but Open Bridge Racks are also easier to maintain and more efficient. “This rack bridges the gap between today’s EIA racks/hardware and the next generation Open Compute racks/hardware,” says Brian Obernesser, vice president data center architecture. “The rack itself is convertible between EIA and OCP standard configurations. Its flexibility enables consumers to convert EIA racks either free or ganged in place to the OCP configuration as they begin to adopt OCP hardware. It also allows integrators or colocation facilities to standardize on a single rack that can respond and adapt to ever-changing customer requirements.”

Fidelity designed (patent pending) and donated it to the Open Compute Foundation, making it available to different manufacturers. The company has standardized on it, using it for its OCP deployments and everything else.

Community involvement

Before designing the Open Bridge Rack, Fidelity partnered with Goldman Sachs and a number of other financial firms to create an Open Compute server that would fit into a standard 19-inch rack. That work resulted in two new specs: the AMD Open 3.0 modular server and Decathlete, for Xeon/Intel.

Members of the Enterprise Infrastructure team have also partnered with other Open Compute participants to start the Compliance and Interoperability Project, which is focused on certification of Open Compute hardware and software. Today Fidelity engineers are working with Open InfraShare, a group focused on how to incorporate open software and hardware into existing architectures. The group meets to share designs and discuss requirements specific to enterprises. The aim is to draw attention and lend credibility to open source solutions.

Throughout the experience, the community itself has been one of the biggest benefits of participating in Open Compute for Fidelity. Bob Thurston, director of integrated engineering has found the community to be a remarkable asset. “The relationships we’ve built through our OCP involvement, especially with peers from other enterprise and hyper-scale data centers, have been invaluable in helping us look at things differently.”

In production

Today, Fidelity is using Open Compute gear to transform its infrastructure. The company is staring to target go-forward platforms, like its private cloud. That’s an environment that allows developers to rapidly build and test critical new apps and features for Fidelity customers—and it’s all based on Open Compute.

Open hardware works for Fidelity because it’s more cost-effective, and also allows the engineers to design for different types of workloads, whether that’s a private cloud or big data analytics. Michael Poulter, senior vice president architecture says, “Open Compute gives us more flexibility to use non-proprietary components in our systems, so that we’re not locked in to any single vendor, and we can more quickly evolve what we’re working on. We want more flexibility, and we think this is a desire we share with a lot of other companies.”

RESULTS
Influence, serviceability, cost savings and sustainability
Despite successive years of double-digit IT growth, Fidelity has achieved a 20% energy reduction across its North American data centers and is ahead of projected optimization targets in data center infrastructure costs. “We have totally changed the way we build data centers in the spirit of Open Compute,” says Joe Higgins, corporate sustainability officer and vice president of engineering. “Open communities are critical in achieving the highest levels of optimization for enterprise data centers.”

Costs have gone down due to less power consumption and also because now Fidelity can use gear from several different vendors who are competing on price. Maintenance is easier as well, which also contributes cost savings. Fidelity’s participation in the Open Compute Project also helps influence the evolution of data center components, shaping the industry as a whole in a way that supports Fidelity’s business.

“Open Compute has helped transform our organization into an ‘enlightened consumer,’” says Alok Kapoor, executive vice president, enterprise infrastructure, “one that’s better educated, aware and self-sufficient. It has empowered our people and firm to take a much more engaged and active part in the technology with a real ROI that goes well beyond cost.”

All the improvements haven’t gone without notice. Fidelity’s leadership understands the benefits as well, embracing open infrastructure as a key element of the firm’s executive-sponsored “compute strategy” initiative. In addition Fidelity has established an open source center of excellence to forward the firm’s broader adoption of open source technologies.

 

To Download this case study click here.

Continue reading this post

Startup firm brings visibility to Open Compute Project hardware performance by collecting and analyzing usage data to help companies get the most from their servers.

Coolan is a data-driven, community-based software platform that provides insight into how data center environments are performing. Founded in 2013, Coolan has two goals: uncovering issues that arise during the lifecycle of a server, and bringing transparency to the enterprise hardware space.

Customers can deploy Coolan on Open Compute Project-designed gear (or any Linux-based server). Like the Open Compute Project (OCP), it draws on the power of a community to help ease the transition to more flexible, efficient, scalable IT. By using real-time information aggregated from a number of data center environments, Coolan’s platform provides customers with actionable data: peer benchmarking reports and recommendations for the configuration, debugging, and optimization of their servers.

CHALLENGE
Measuring data center performance

Before Amir Michael founded Coolan, he worked as part of the Facebook data center team that kicked off the Open Compute Project in 2011 by redesigning the company’s servers and open sourcing the design. “The Open Compute Project started a conversation,” Michael says, “about the lack of transparency within the data center industry, and how data center operators could best work together to reduce operational complexity and increase efficiency.”

As part of that conversation, Michael spoke to CIOs, engineers and operations teams at many organizations interested in deploying OCP servers. He noticed that few of them effectively collected metrics on the performance of their infrastructure. And even if they did collect it, they often weren’t analyzing it as much as they could have been. “There’s a lot of untapped value there,” he says. “This data can tell you if you’re operating at full potential, and if not, how to get there.”

Large organizations are better equipped to optimize their hardware based on the sheer scale of their deployments—their data sets are generally bigger, and they have the resources to analyze and leverage them. But, smaller firms often don’t have the resources or a big enough data repository to do it. “The information exists,” Michael says, “but it’s not readily accessible.”

SOLUTION
A community-based data set

Coolan makes the data accessible, by combining input from a community of participants into one large, shared set. “We bring that large data set to anyone with a server,” says Michael.

Coolan’s customers use servers from many different vendors—including OCP solution providers. They send their data to Coolan, whose automated software analyzes it based on thousands of operational variables, from how many hours a component has been in operation to how many bit errors have been generated by memory.

Coolan then takes it one step further, aggregating all the data it receives, making it anonymous and sharing its findings on server configuration, operating temperature and failure rates with the wider community. Companies that participate receive insight into industry benchmarks, the most stable server configurations, and more. “We extract a lot of information about server stability and efficiency to establish trends and even predict what might happen,” Michael says. “While there may be some initial hesitation about sharing data, we have found that people decide to participate because they realize they will get value in return. They can analyze their own infrastructure and compare to the community to gain a clearer picture of future equipment failure.”

Coolan is in the early stages of development, but has already begun to help make the hardware purchasing process more transparent for enterprises. Previously, data about server performance were dependent on a vendor’s own marketing claims; Coolan serves as a neutral, vendor-agnostic source of information that can help reduce downtime and ultimately, lower the cost of infrastructure. “We are working with companies that are giving us data and helping guide our product development,” Michael says. This ranges from firms with several hundred servers to one with more than 500,000.

RESULTS
Better insight, better decisions

Coolan’s platform is bringing more transparency to the server industry, and showing potential to help companies determine if a move to OC hardware is right for them. “Some in the industry view OCP with skepticism,” says Michael. “They want it because it’s more economical and more flexible, but they’re afraid because there’s no safety net.” By collecting and disseminating data on failure rates, Coolan is adding transparency to the process of purchasing hardware. Customers can see how servers from different vendors—whether they adhere to OC designs or otherwise—stack up in terms of performance. Michael says, “We are giving companies the means to make an informed decision on whether they want to stick with traditional vendors or start migrating to OCP.”

Michael says OCP has helped to create a new paradigm for how the industry looks at hardware. “The momentum behind OCP clearly indicates that people want more efficient hardware. We are changing what used to be a very opaque, proprietary, don’t-look-behind-the-curtain business where vendors controlled every bit of information about operations. Now, the data center industry is opening up and increasingly driven by an engaged community of hardware developers and consumers. OCP has inspired people to rethink how they approach servers and data centers,” he says. “Our principles mirror those of the Open Compute Project: transparency, building a community and putting control in the customer’s hands.”

Continue reading this post

Today, Penguin announced that the U.S. Department of Energy's National Nuclear Security Administration (NNSA) has announced that it plans to deploy a series of OCP-inspired high-performance computing clusters designed by OCP solution provider Penguin Computing. The systems will provide additional computing resources for the national laboratories at Los Alamos, Sandia, and Lawrence Livermore.

There is still more that can be done to push the industry toward greater efficiency, larger scale, and better flexibility, but today's news showcases the momentum we're seeing from the community and the progress the industry is making toward open hardware.

Looking to get involved with OCP? Find out more here.

 

 

 



Continue reading this post