Before you enter the data science industry, you might have wondered if you are looking to work for a startup company, where you will wear many hats and have to figure a lot of processes on your own, but also get the freedom to explore; or enter a corporate company, where there is a large amount of data and structures for readily for you, but may limit your growth or creativity.
So how do you even decide?
Today, we have Wai Kit, who has worked in both – he spent 2 years working in Fave, a fast-moving local startup company; and now he is the data science team lead in GBG, a corporate data intelligence company.
He shares with us his journey in both, and how he adapts in both working cultures, and move forward in leading his team.
What was your role in Fave?
During my time in Fave, I was working on exploring a lot of data work, as at that time we were looking to scale the company to the next level. We spent a lot of time understanding and analysing our customers and merchants’ data points, so we can gain better insights on their behaviours and that largely contributed to the growth of the company.
Moving from Fave to GBG, what are the key differences of working for both these companies respectively?
There are definitely different challenges faced in both companies. However, there is not a stark difference in the nature of the work we do.
Ultimately, the goal of the company’s data team is always to enhance the operating processes, or tap into new opportunities from a data point of view.
But what I always try to do, is to incorporate whatever I have learned and the insights I have gained from Fave and put it into the work I do at GBG.
From a data engineering point of view, is there a stark difference between the amount of data that you handle in Fave versus the amount of data you handle in GBG?
GBG would definitely be comparatively more challenging in terms of data pipelines. This is because there are many more different kinds of data sources and data servers that we are working with, to which we are required to connect into our data warehouse.
How did you progress from being a data engineer into a data team lead, And what are the key differences in the responsibilities of both the roles?
When I was in Fave, I have actually got a taste of leading a data team. When I first moved into that role, I realised that it truly was my interest to lead a team. I enjoyed managing a team consisting of various different talents like data scientists and data engineers, and even people from business intelligence.
When you are a team lead, the way you work becomes very different because the job is no longer about you, but rather, it consists of the work of the entire team.
What are you and your team’s current day to day tasks? And what are the common tools you use to execute these tasks?
Our main work consists of collecting data from various different sources, and we work a lot on ETL (Extract, Transform and Load) data work, and then storing it into the data warehouse.
It sounds pretty straightforward, but the majority of the time is spent on cleaning the data. It takes up yo 70% of our team’s time to clean the data so it is ready enough to be analysed.
In regards to the tools we use, especially for data engineers, it is pretty much flexible on what tools we use as long as are able to carry out the processes. One of the most used tools would have to be Python and SQL, but we also do Shell Scripting.
How many people do you manage in your team? And how do you do it?
Currently, I am managing a team of 12, which will soon become a team of 14. When I was back in Fave, I was only managing a team of 7.
The way we manage a team of 12 is very different from the way we manage a team of 7. This is because when your team is bigger, the team members will be required to work more independently, as you would not have the capacity to oversee every single person’s day to day work. Besides, I never believe in micro-managing people.
As a team lead, aside from technical skills, what is the most important trait you look for in an individual to join your team?
I personally think the most important trait of a data team member is passion. Only with true passion and love for all things data-related, you are able to fully excel in this industry.
Especially if you are working in data engineering, it requires a lot of long hours working on behind-the-scenes processes such as cleaning the data in order to lay the groundwork for the next courses of action. If you are not passionate about data, you will not be able to endure the daily work tasks and responsibilities required of you.
Catch Wai Kit’s presentation in D/M Summit: How To Prepare A Manufacturing Plant for Industry 4.0
How do you manage a team of individuals with different personalities or working style?
One of the first things I did in my first week of being a team lead in GBG, was holding a retrospective session with the team.
This session encouraged the team to look into their own work respective to the contributions to the team, and we explored the necessary improvements needed, addressed their personal concerns for the team etc.
From there, we worked towards an agreeable consensus to see how we can come up with a solution that everyone is comfortable with.
With a lot of highlight on the Industrial Revolution 4.0 (IR 4.0), how do you think it is impacting Malaysia?
To be honest, as of now, I think Malaysia is still at IR 3.0, but we are indeed slowly progressing towards 4.0.
In order for us to move into the next stage, it is very important for us to empower all levels of organisation and executives to embrace data into their day to day work.
With IR 4.0, it will create more high-level functioning job opportunities, especially in the data industry, as the mundane and laborious work will be automated.
What is the most important reason why companies want to move into IR 4.0?
Companies should look into automation with the help of AI as a means to move into IR 4.0. This is because this transition can largely improve the company’s Return of Investment (ROI). When processes can be replaced by automation, it will help reduce operating cost, and at the same time increase the output without increasing the resources.
How can smaller organisations with lesser budget start to explore and adopt IR 4.0 into their companies?
There are actually many readily available services out there such as Amazon or Google, where they do not charge based on the size of the company, but rather the number of users, or the required size of cloud or server.
So these platforms are considerably affordable for companies who are still testing out and are looking for a lower budget solution.
What is your main tip for people who are looking to start adopting IR 4.0?
The most important step is for you to start collecting data. Because without data, there is no possibility for you to even explore any form of data intelligence. Once you have a significant size of data, you will be able to explore connecting your data and find your way to progress from there.
With Wai Kit’s experience, he also shares more of his thoughts on how companies can move into IR 4.0 more seamlessly in his session. If you like to learn more from him, or from other industry experts on data science and AI, be sure to grab the Lifetime-Access Pass to tune in to the D/M Summit!