Hadoop Infrastructure Engineer - Foundational Services

• Bloomberg L.P.,

Analysis, Automated, Business Intelligence, Configuration Management, Data Analysis, Database, Developer, Development, Hadoop, Hardware, Java, Linux, Management, Networking, Python, Ruby, Security, Shell Scripting, Testing, Unix

Full Time

Telecommuting not available Travel not required

Job Description

Job Requisition Number: 49874




Many of the most prevalent applications in Bloomberg need real-time data analysis and business intelligence. Instead of using off-the-shelf applications, we use Hadoop and it's ecosystem to provide large-scale data platform with low-latency SLAs and large storage capabilities. This platform has revolutionized the way we manage and analyze data in a distributed environment.


We design software and hardware systems to support low-latency/high-volume requests, security, fault tolerance/high availability and easy customization. Our Hadoop Infrastructure Platform is built to fully automate deployment and operations using Chef; developed and open sourced at https://github.com/bloomberg/chef-bach . With hundreds of applications depending on our platform, we are looking to grow our Hadoop Infrastructure team. That's where you come in.

We'll trust you to:

• Evaluate Hadoop projects across the ecosystem and extend and deploy them to exacting standards (high availability, big data clusters, elastic load tolerance)
  
• Develop automation, installation and monitoring of Hadoop ecosystem components in our open source infrastructure stack, specifically HBase, HDFS, Map/Reduce, Yarn, Oozie, Pig, Hive, Tez, Spark and Kafka
  
• Dig deep into performance, scalability, capacity and reliability problems to resolve them
  
• Create application patterns for integrating internal application teams and external vendors into our infrastructure
  
• Troubleshoot and debug Hadoop ecosystem run-time issues
  
• Provide developer and operations documentation to educate peer teams
  

You'll need to have most of the following:

• Experience building out and scaling a Hadoop-based or UNIX-hosted database infrastructure for an enterprise
  
• 2+ years of experience with Hadoop infrastructure or a strong and diverse background of distributed cluster management and operations experience
  
• Experience writing software in a continuous build and automated deployment environment
  

We'd love to see:

• 2+ years of DevOps or System Administration experience using Chef/Puppet/Ansible for system configuration, or quality shell scripting for systems management (error handling, idempotency, configuration management)
  
• In-depth knowledge of low-level Linux, UNIX networking and C system calls for high performance computing
  
• Experience with Java, Python or Ruby development (including testing with standard test frameworks and dependency management systems, knowledge of Java garbage collection fundamentals)
  
• Experience or exposure to the open source community (a well-curated blog, upstream accepted contribution or community presence)
  

We want to work with others who are passionate about community-driven development both within the company and with the wider open source community. If this sounds like you, submit an application, and learn more about the work we do from Clay and Amit's interview at ChefConf2015: https://www.youtube.com/watch?v=LnMCFxXgDE w




Date: Wed, 25 05 2016 00:00:00 GMT
Department: Software Developer/Engineering

Email this job to yourself or a friend:

From my email address:

Recipient's Email

---

Company Information

Bloomberg is a company dedicated to helping solve complex challenges through insight and information. Our strength—quickly and accurately delivering data, news and analytics through technology—is at the core of everything we do. With over 15,500 employees in 192 locations, we give influential decision makers in business, finance and government a competitive edge by connecting them to a dynamic network of news, people and ideas. To do so, we need constant energy and innovation—which is where you come in. At Bloomberg, you will have the opportunity to take risks and be part of an organization that is entering new markets, launching new ventures and pushing the boundaries. Our ever-expanding technology, data, news and media services afford employees the opportunity to expand skills and connect with smart, driven colleagues from a diversity of backgrounds and ideas. We're looking for dynamic, multi-talented people who have a desire to thrive in a forward-thinking culture and a business with global impact. Are you ready to make your mark? Learn more about our businesses and opportunities at bloomberg.com/careers.