Summary

Expertise

Project Highlights

Education

Agency

LF

English:

Upper Intermediate

Luciano F.

vetted by Youteam

Vetted by YouTeam

Brazil

UTC -03:00

America/Fortaleza

English:

Upper Intermediate

Experienced Data Engineer proficient in SQL, Python, Spark, Databricks, and Azure Data Services. Skilled in ETL pipeline creation, prioritizing clear

- Leading and developing the Data Warehouse Migration project to a Data Lakehouse on Azure, demonstrating leadership and technical expertise. - Implementing data governance improvements, including a Data Catalog service and observability tool. - Reducing the average execution time of the main ELT pipeline in the legacy environment, showcasing optimization skills and the ability to improve existing processes. - Building predictive models for estimating freight trains' transit times and improving forecasts, indicating advanced analytical and modeling skills.

Want to hire this engineer?

Check if Luciano is available

Expertise

Years of commercial development experience

12 years of experience

Core technologies

Data Warehousing 8 years
Python 8 years
SQL 12 years
Azure 5 years
ETL 8 years
Databricks 3 years

Other technologies

R
Spark

Project Highlights

icon
Principal Data Engineer

Grupo Recovery

Dec `21 - Oct `23

2 years

Data Warehouse Migration project to a Data Lakehouse on Azure with Python and Spark.

Responsibilities & achievements

* I developed and deployed the structural parts. Currently, I am working on data cleaning and transformation with two more data engineers whom I have been training. * Boosted Spark Jobs' performance on Databricks by optimizing queries, data caching, table partitioning, cluster resizing, query improvements, resulting in a substantial reduction in execution time and computational costs. * Improved data governance in the company's Big Data environment by implementing a Data Catalog service (Unity Catalog), an observability tool for clusters and jobs (Overwatch), and personalized access controls for internal and external data based on users and areas. * Created data quality validation and data cleaning processes that drastically reduced the ticket opening rate related to data quality. * Reduced the average execution time of the main ELT pipeline in the legacy environment from 12 to 7 hours by reformulating the orchestration, tuning SQL procedures, and identifying and removing unnecessary steps.

Azure
ETL
Python
Spark
Data Warehousing
Databricks
icon
Senior Data Scientist

Dti Digital

Apr `21 - Dec `21

8 months

Predictive models that provide real-time estimates of freight train transit times on a logistics company’s railway network.

Responsibilities & achievements

Completed a predictive time series analysis project, which delivered a 12% improvement in refractory material consumption forecasts.

Machine Learning
icon
Data Engineer

Belgo Arames

Nov `19 - Mar `21

1 year

ETL pipeline for steel importation data from external sources and designed data models that integrated with information extracted from the CRM.

Responsibilities & achievements

•Developed a model that identifies possible tax evasion fraud on national steel imports. In the first 6 months, more than 30 confirmed cases of fraud were identified by the model. •Joined the company's Innovation Ambassadors team, becoming point for data solutions

ETL
icon
Data Scientist

Usiminas

Feb `19 - Oct `19

8 months

Model used to adjust sales strategies, budget definitions, and purchases by the Supply Chain team.

Responsibilities & achievements

- Improved the visibility of growth and fluctuation of steel demand by customers, with a predictive modeling of Machine Learning. - Developed a People Analytics project with the HR team, helping them to identify professional profiles and control the turnover of more than 14 thousand employees.

Machine Learning

Education

Higher Education

Agency

Near-Shore staffing agency #2268

100-400

GMT-3

Fortaleza/Brazil

Core Expertise

AngularJS
AWS
Cloud Engineer
Django
GitHub
JavaScript
Node.js
Python
React.js
React Native
TypeScript
Angular 2x
Bitbucket
Next.js
Terraform

Industries

E-Commerce & Retail, Banking & Finance, SaaS

Want to hire this engineer?

Check if Luciano is available