Python Data Engineer

140 - 160 PLN/h netto + vat (B2B)

100% zdalnie

Regular / Senior

Full-time

• Firma:

  Bertek Polska sp. z o.o.

• Wielkość firmy:

  11-50

• Zespół:

  Brak informacji

Wymagania

• Senior

  • Python

• Regular

  • SQL
  • NoSQL

Minimalna liczba lat doświadczenia w IT: 5

Opis pracy

We are looking for Data Engineer, who will join Data Team works on building and deploying novel analytical algorithms based on the data from IoT products. The person in this position will work on the full stack; from data pipelines and processing to creating user facing applications.

Your responsibilities:

● Build and run Azure based Data Platform.

● Handle multiple Terabytes of different domains data with focus on IoT data using Stream and Batch processing.

● Craft and maintain world-class, high traffic analytical applications and services with a team of experienced data engineers and scientists.

● Develop data models and process raw data at scale to make it available for analysis.

● Build scalable data pipelines using state-of-the-art technologies in the Microsoft Azure Cloud.

● Implement, run and deploy algorithms in cooperation with data scientists using the Hadoop ecosystem (e.g. Spark, Hive, HDFS) and the Microsoft Azure Stack).

What we expect:

• At least 5 Years of experience in architecting and building Data Lake and Enterprise Analytics Solutions
• Ability to optimize 'Big Data' data pipelines, design Data Lake architecture and data sets
• Experience with Design and Architecture of relational SQL and NoSQL databases
• Advanced hands-on SQL, Python, pySpark and HDFS knowledge and experience working with relational databases for data querying and retrieval
• Experience with Design and Architecture of Azure big data frameworks: Azure Databricks, Azure Data Factory, Azure ML, SQL Data Warehouse, HDInsights, Stream Analytics, Azure Data Explorer - or their open source counterparts: Kafka, Spark, Blob Storages.
• Knowledge of DevOps Processes (including CI/CD) and infrastructure as a code fundamentals
• Knowledge of Data Quality issues and methods of dealing with them
• Fluent in English

Specyfikacja pracy

• Praca zdalna