BE-EN 
EN
NL
DE
BE (NL)
Felyx's Data Transformation Journey: Unveiling the Secrets of Sustainable DataOps
In the ever-expanding realm of modern organizations, data reigns supreme. To harness its full potential, a well-crafted data strategy and a flexible data structure are essential. Felyx, the Dutch e-scooter rental company, has embarked on a remarkable DataOps journey, overhauling its data infrastructure. In this blog, we delve deep into their experience, revealing valuable insights for data specialists and enthusiasts alike.
Lesson 1: The Power of Team Building
Choosing the Right People
Felyx's Global Head of Data Analytics, Daan Stroosnier, emphasizes the significance of assembling the right team. With a shortage of skilled technicians in the industry, the selection process plays a pivotal role. Stroosnier highlights the importance of integrating this aspect into the project's planning, acknowledging its potential impact on decision-making and overall project success.
Lesson 2: The Art of Integration
Not Just for Systems, but for People Too
Intricately woven data systems require harmonious collaboration among diverse teams. Stroosnier advocates for fostering cohesion within autonomous teams to unlock synergies and enhance the overall project effectiveness.
Lesson 3: Balancing Resources
In-House Talent and Freelancers
Striking a balance between in-house employees and freelancers is crucial for knowledge transfer. Felyx benefited from specialized freelancers initially, but Stroosnier cautions against over-reliance, emphasizing the importance of continuity in knowledge-sharing.
Lesson 4: Agile Realism
Scrum with Pragmatism
Data engineer Mees Strooker underscores the popularity of Scrum and other agile methodologies but advises teams to maintain a realistic approach. Aligning internal goals with company deadlines, Strooker advocates transparent communication and proactive engagement with other teams.
Lesson 5: Versatility is Key
The Flexibility to Adapt
Felyx's unwavering commitment to adaptability and foresight guided their tool selection. Strooker emphasizes the need to consider consequences before implementing tools, enabling the freedom to pivot if necessary. Their modular approach and commitment to the Unix philosophy empowered them to make informed choices.
Lesson 6: Scalable Foundations
Thinking Ahead with Data Models
Stroosnier highlights the hindsight-driven need for a more scalable data model. Felyx's pragmatic approach in the early stages ultimately led to considerations for future scalability, reminding us of the importance of foresight in data architecture.
From Legacy to State-of-the-Art
Legacy: The Old Way
The original data infrastructure at Felyx grew organically but lacked scalability. The PostgreSQL database struggled with performance issues. Debugging, fault-trapping, and validation were poorly documented, making it challenging to backtrack or rectify errors.
New Situation:
A Fresh Start
Felyx embarked on a transformative journey, emphasizing key principles:
- Scalability and elasticity
- Embracing change
- Applying the Unix philosophy
- Improved debugging
- Transition from ETL to ELT
Their new approach stores raw data in various cloud buckets, employs Kubernetes pods for event data processing, and manages schema using Liquibase.
Data Model: Building for Quality
Felyx employs a layered data model to ensure data quality and usability for analysts and scientists. The choice of a 3rd normal form with history allows for enriched layers on top of normalized tables, promoting flexibility and adaptability.
Data Processing: Streamlining Insights
Data Build Tool (DBT) facilitates data model management and data mart updates. Data analysts transition into the realm of data engineering, with version control, testing, and reusable code.
End Users: Meeting Diverse Needs
Felyx serves a diverse user base, including:
- Analytics translators, who use Metabase for descriptive and diagnostics analysis
- Analytics engineers, creating custom dashboards in RShiny and Dash
- Data scientists, leveraging ML models for predictive and prescriptive analysis
The End Result: A Transformed Landscape
Felyx's dedication to flexibility, comprehensive change management, and robust documentation paid off. Their choice of tools and a data model that supports schema modifications as needed, sets them up for success in an ever-evolving data landscape.
Conclusion
Felyx's DataOps adventure serves as an inspiring journey for data specialists and enthusiasts, offering a multitude of valuable lessons and insights. The transformation of their data infrastructure underscores the importance of adaptability, team dynamics, and scalable models in the data-driven world. As we embrace the future of data, Felyx's story stands as a testament to the power of a well-executed DataOps strategy.
For organisations
In need of Intelligence, Data or Analytics professionals? Visser & Van Baars gets you to the next step with our extensive network of experts. Read more about our IT staffing & Consulting services for organisations.
For professionals
Want to boost your career? Visser & Van Baars is the partner that helps you to your next assignment or employer. Read more about the possibilities and find your match.