Written by —
How to stay up-to-date with the newest technologies, tools, and architectures? This is a question we at Recordly are resolving with our Hackathons, the ultimate learning platform. In this read, you will get a peek into our latest Hackathon focusing on the data build tool (dbt) and have the opportunity to learn more on how to use it.
Written by —
Staying on top of all the new technologies, tools, architectures, and ways of working can be a daunting task. In addition, as a consultant, you should be ahead of the curve, knowing and having experience of the latest and hottest technologies while doing productive client work at the same time. And all should be done in roughly 37,5 hours per week.
To top things off, being a fresh new data consultancy business, we are figuring out and testing how to run things; how to organize our IT department, what benefits to offer and how, how to build our culture, how to do recruiting, and what areas should we expand to?
Hence, the big question is; how to stay ahead of the game with new technologies by having hands-on experience in them, be you, and build a rocking data company?
At Recordly, we're solving these questions by organizing Hackathons every month.
Hackathons at Recordly
Our Hackathons are one to two-day events, during which we gather all of our troops together to learn about new technologies and tools, review business cases, and plan and improve our company.
The idea is not only to keep learning and developing ourselves and our company, but also to take time off from busy project work and spend time together. As consultants, developing ourselves as professionals is a given, but it is equally as important to unwind together and do something else than billable work. For example, by spending time together and wrapping our brains around new technologies, challenging business cases, and developing our company and culture.
"Our Hackathons are one to two-day events, during which we gather all of our troops together to learn about new technologies and tools, review business cases, and plan and improve our company."
Our Hackathons have two tracks - one technology track and one company track - and the track's subject is flexible. Want to tell someone about that sweet Jinja-syntax for looping through items in dbt and Airflow, or did you try out Firebolt and have some first-hand experience and ideas about it? Technology track! Do you have an idea how the company could create a framework for personalizing benefits or do you have something to say about how people are allocated to projects? Company track! Have an interesting event-based architecture with Confluent at the core? Invite the sales engineer from Confluent to be at the Technology track! The opportunities and topics are endless.
Arranging our dbt Hackathon in Tampere
For our Hackathon, we arranged a hands-on session to deep dive into dbt, as the tool seems to be becoming more popular. I decided to volunteer since I have some experience planning, implementing, and running dbt based projects, but more importantly “the best way to understand a concept is to explain it to someone else”. So I planned a one-day workshop/training around the data build tool, which focused on the following goals:
- the basic idea about what dbt does,
- what it doesn't do,
- how to set it up and start using it,
- how to expand it with packages and macros,
- and what to consider when creating a production-ready environment.
During and after the hands-on session we had long discussions on positioning dbt in the modern data stack and how it compares to other tools. Based on how our discussion evolved throughout the day, it is safe to say that we achieved the goals! And with a cherry on top, we also got our CEO, Katriina, to actually run dbt from the command line! 😱
Why focus on the data build tool?
Why should you then learn something about data build tool? If you google "Modern data stack", 9/9 front page hits mention dbt as part of or as an example tool in the modern data stack. So clearly it is something to take note of.
dbt is a tool for managing the transformations inside a cloud data warehouse. dbt core is an open-source CLI tool with support for postgres, Redshift, BigQuery, Snowflake, Apache Spark, Databricks, and Presto out of the box. Azure Synapse is supported by the open-source community.
Besides the transformations, you can do tests, documentation, orchestration, and SQL generation via macros. What you can't do is integrations or deployment management — you'll need to build your own practices for CI/CD.
To learn more about dbt, ensure to check out our dbt hackathon repo.
Join recordly and our hackathons!
If you want to participate in our upcoming hackathons to develop your data capabilities and take part in building Recordly, do check our open positions and send us your application.
