Disclaimer: I work at Cumulus Networks
To address a few of the high level points/concerns raised:
1.) Is ONIE getting traction?
ONIE is the de facto method for getting software onto whitebox switches. It's been embraced by Dell, HP, Supermicro, Penguin, Facebook and a few others. It's run out of the Open Compute Project (http://www.opencompute.org/) an independent organization working to make datacenter equipment more open, more affordable and more interoperable. When this post was first written HP was mentioned; they didn't have open networking switches. Today HP has open networking switches with ONIE that run Cumulus.
2.) Is it stable? Can I trust it?
Sure. Cumulus is running some of the biggest clouds on the planet, as well as hospitals, police departments, school districts, financial services firms and everything in between. I'm sorry that you had a crash, but with every Cumulus License comes 24x7x365 support based in North Carolina, USA and Sydney, Australia. If you run Cumulus and run into problems they are happy to help!
3.) Coming from a Cisco/Arista background, I don't know what to think of this Linux stuff.
It's different and that can be a challenge for sure. Using FRR to configure BGP or OSPF on Cumulus will look very familiar to those with a Cisco background. To configure IP addresses, bonds (etherchannels), mLAG (vPC) or other L1/L2 features we have a lot of documentation to help you out
and again, if you get stuck you can try to search "how to configure an IP on Linux" and it's the same configuration for Cumulus, or call support!
4.) The staff doesn't have the skills they need
That's probably why they called in reinforcements for help on the networking side of the house. Personally, I come from a Cisco background and I think that network engineers and architects have a very important skill. We build networks that are scalable and fault tolerant. That's a skill a lot of server people do not have. But do we need CCIEs to configure VLANs or modify an ACL on day to day operations? I don't think so. Let's have network engineers build networks and systems/ops people operate systems (+ networks).
If there's still a gap or concern we offer full day bootcamps to help get you comfortable, or you can use our virtual edition, Cumulus-Vx (www.cumulusnetworks.com/cumulus-vx) and get a feel for how the software behaves.
If you're interested in hearing more, we'd love to win you back! The things you can do with Cumulus from an automation, configuration, choice and price standpoint make us a great product.
Pete Lumbis - CCIE #28677, CCDE 2012::3