Job description
About us
Wefarm, the world’s largest farmer-to-farmer digital network, enables farmers to connect with each other and key partners over SMS to solve problems, share ideas, obtain vital products and services, and spread innovation, through utilising the latest machine learning technology.
Small-scale agriculture is the biggest industry on earth, with more than a billion farmers globally supplying 70% of the world's food and commodities, yet remaining digitally unconnected. Until Wefarm, no other had built a digital platform for these farmers to share their vital insights, without having to go online!
Since our launch in 2015 we have grown to serve 1.9 million farmers across the world, who share more than 30,000 Qs & As per day. Wefarm has recently secured $13 million in Series A funding from some of the world’s leading VCs, including True Ventures in Silicon Valley, and we are looking to add to a world-class team based across London, Nairobi, Kampala and Dar-es-Salaam.
Join us and be a part of our mission to build an ecosystem for global agriculture, with the farmer at the centre!
The role
We are looking for a Data Engineer to join our growing technical team, as our first full-time hire in this field. You will have responsibility for architecting and managing new data management processes, including data schemas and ELT (extract-load-transform) data pipelines. You will lead on greenfield projects, owning research and development, as well as having the remit to optimize our existing infrastructure.
The role sits within Wefarm’s Technology team and will report directly to the CTO.
Wefarm currently sends and receives millions of SMS per month, leading to tens of millions of independent system events. You will be responsible for the storage and transfer of this unique dataset. You must be self-directed, and comfortable supporting the data needs of multiple teams, including developers, BI, and data scientists, enabling Wefarm to use data to build insight, grow the company, and support the needs of our farmers.
There is a rich, vibrant, and open culture within Wefarm. Even though we are separated geographically, we maintain a highly collaborative and communicative outlook and are keen to maintain this as we expand into new roles and new territories.
Essential Duties:
- Build and maintain ELT (extract-load-transform) pipelines to get data from various systems and databases into our existing Snowflake data lake
- Build and maintain SQL (and non-SQL) data transformations to enrich our data, including using DBT as the transformation step within our data lake
- Enable stakeholders, including the BI, Product, and Growth teams, to get access to the data they need to perform their duties
- Collaborate with Product Development teams to research and develop new data infrastructure projects, and architect reliable, accurate, and useful data schemas which can work for both operational and reporting purposes
- Work across the company on data-related technical issues, and monitor database usage and performance to maximise efficiency
- Ensure data security and data protection regulatory compliance by implementing optimal access policies and following industry best practice
- Razor-focused on personal and team OKRs to ensure work is aligned with company goals
Job requirements
- 3+ years experience working in a Data Engineer role (or equivalent experience in a backend engineering role)
- A successful history of manipulating and processing large disconnected unstructured datasets, and enabling others to use data to build value
- Advanced SQL (PostgreSQL and Snowflake)
- Working knowledge of Git and GitHub
- Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores (e.g. SQS, Kafka, Snowflake)
- Strong project management and organizational skills
- Experience supporting and working with cross-functional teams in a dynamic environment
- Able to clearly articulate and communicate with non-technical team members
- Desirable: Clojure, Python and/or R
- Excited about Wefarm’s mission