I completed work experience with Sentium Consulting in 2022. I was tasked with creating and implementing a data pipeline app in Python to take in big datasets from external APIs, clean & transform the data, and store it in a database. The workflow needed to be managed and tracked by Prefect, for ease of debugging and tracking the data.
I gained experience working to external deadlines and delivering a product to an assigned spec. I also worked in an Agile Workflow Environment and developed skills composing Python and working with large datasets, all of which were new to me at the time.
Overall, the project went well and I performed the task to the required standard; this was a great learning experience for me.
