2015 – Now
Middle Software Developer
LogBroker had became the de-facto standard for internal data transfer and currently spans multiple data centers in different geographical regions and hundreds of nodes.
As a middle developer I had continued developement of important LogBroker features including:
2013 – 2015
Junior Software Developer
Currently I'm working on LogBroker project – large distributed system based on Apache Kafka. LogBroker serves as a centralized data bus, which controls internal company's data flow, reaching hundreds of Terabytes of throughput on a daily basis. It aggregates data from servers across the company, preprocesses it to unify format and delivers to the processing endpoints such as MapReduce clusters for batch processing or real-time log analytics systems, while ensuring lack of data loss and duplication (exactly once delivery semantics).
LogBroker is being successfuly used in production environment for over a year and gradually replaces old data flow infrastructure. 95% of data travels end-to-end from server of origin to processing endpoint in less then 3 seconds with goal towards reaching 99% mark.
During my involvment in the project I:
In the course of development I've had expirience with Scala programming language, Akka and Spray frameworks, Apache Kafka and ZooKeeper projects. I've also learned a lot about building fault-tolerant systems and distibuted service architecture.
2012 – 2013
Intern Software Developer
Worked on MapReduce jobs for ad hoc analytics written in Perl and Python.
2010 – 2014
Bachelor of Information Technology