Summary

Expertise

Project Highlights

Education

Agency

AQ

English:

Upper Intermediate

Andy Q.

vetted by Youteam

Vetted by YouTeam

Peru

UTC -05:00

America/Lima

English:

Upper Intermediate

Strong Senior Data Engineer with more than six years of experience.

Strong Senior Data Engineer with more than six years of experience. I have a diverse background where he has established Data architecture solutions and he’s also led teams using agile methodologies. Strong experienced in relational and non-relational databases, Scala, Python, Spark, and AWS Stack (EMR, Redshift, S3, CloudFormation),GCP, and Azure amongst others for ETL and data pipeline-related work, and distributed file Systems(HADOOP)

Want to hire this engineer?

Check if Andy is available

Expertise

Years of commercial development experience

6 years of experience

Core technologies

Python 8 years
pandas 6 years
SQL 8 years
PostgreSQL 7 years
AWS Lambda 7 years

Project Highlights

icon
Senior Data Engineer And Data Architect

NTT DATA

Built automations and extract data from (salesforce marketing cloud and salesforce sales cloud)

Responsibilities & achievements

● Extract data from third party API´S ● Build pipelines to merged data in the data architecture ● Build dashboards and reports ● Apply improvements to the existing pipelines ● Develop end to end pipelines ● Pull data from different sources(sql, file systems, ftps,etc) ● Delivery included: Google Storage, Airflow, Google Functions, Cloud Scheduler , Python , Dataproc , Dataflow , Cloud composer , Salesforce Api, Snowflake, MongoDB, Kafka, Pyspark, Sql server, Tableau

Google Storage
MongoDB
Python
SQL Server
Pyspark
Snowflake
icon
Senior Data Engineer And Data Architect

Spark Digital

● Built automations and extract data from (salesforce marketing cloud and salesforce sales cloud)

Responsibilities & achievements

● Extract data from third party API´S ● Build pipelines to merged data in the data architecture ● Build dashboards and reports ● Apply improvements to the existing pipelines ● Develop end to end pipelines ● Pull data from different sources(sql, file systems, ftps,etc) ● Delivery included: Google Storage, Airflow, Google Functions, Cloud Scheduler, Big query, Python , Dataproc , Dataflow , Cloud composer , Salesforce Api

Google Storage
Python
Salesforce
icon
Senior Data Engineer and Data Architect

Everis Peru

Banking(Credit Bank of Peru,Pichincha,Interbank), Assurance(La positiva),Telco(Telefonica)

Lead and built applications with big data technologies within data lake architecture

Responsibilities & achievements

● Provided technical direction and leadership to project teams ● Worked with COE (Center Of Excellence) - BCP in DevOps, DataOps, Machine Learning and Big Data projects ● Created and led POCs to stay up-to-date with advances in data persistence and big data technologies and run experiments/POCs(aws,azure,gcp) ● Build computer visions with security cameras and getting information from sensors ● Built Dashboards with kpi´s ● Designed and proposed architectures to new jobs(aws,azure,gcp) ● Created task to execute sql queries ● Made troubleshooting to existing jobs ● Migrated business logics from sql to pyspark ● Created ETL flows into SFMC ● Extracted information from API´s (WeAreHearken) ● Built complex data pipelines for data processing into datalake architectures(spark) ● Built data pipelines to extract information from different sources and merge the data into same architecture(datalake) ● Deploy virtual environments ● Delivery included: Oracle, SQL Server, Pyspark, Scala, Spark,Hadoop, Hdfs, Hive, Impala, Cassandra, AWS Stack(EMR, Redshift, S3, CloudFormation), ElasticSearch, Kafka, SparkStreaming, Sqoop, and Nifi,OpenCV,NLP,OpenFace,Tensorflow,Salesforce marketing cloud, Salesforce Sales, Airflow, GCP stack(Google Functions, Cloud Scheduler, Dataflow.Google Data Studio, Terraform, BigQuery, Google Storage), Azure(Azure datafactory, azure databricks,cosmodb,storage account, azure datalake,Keyvault, Service Principal(app registration),azure functions, azure sql server,Azure devops,azure synapse analytics, power apps, power automate)

Azure
Cassandra
Oracle
Salesforce
Scala
Spark
SQL Server
Pyspark
icon
Senior Big Data Engineer

Fit Big Data

Client: Banking(Goldman Sachs)

● Developed applications for batch processing

Responsibilities & achievements

● Developed applications for batch processing with Scala and Spark ● Optimized and applied improvements to existing applications ● Built pipeline for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and big data technologies ● Built analytics tools that utilize the data pipeline to provide actionable insights ● Built ETL for data extraction from the sources ● Built automations and extract data from (salesforce marketing cloud and salesforce ● sales cloud) ● Built and improve accuracy machine learning models (Linear Regressions, decision ● tree), NLP ● Web Scraping and collect data into nosql databases and distributed files ● Delivery included: Qlik Sense, SQL Server, Oracle, Nifi, Spark, Scala, Pyspark, PysparkML,S3, EMR, Athena, Redshift, QuickSight, Kafka, SparkStreaming, Hadoop,Hdfs,Sqoop, Salesforce,Glue, Lambda, marketing cloud, salesforce sales cloud, Airflow, Jenkins,Bitbucket, Fortify

Scala
SQL Server
Jenkins
Lambda
Pyspark
icon
Big Data Engineer

Everis Perú

Client: Banking(Credit Bank of Peru)

● Built batch processing pipelines ● Built dashboards with logs/metrics consumption and analysis

Responsibilities & achievements

● Worked together with Cloudera team to solve and report issues ● Built complex batch and real time jobs ● Extract data from different sources ● Migrating business logics from sql querys to spark Built dashboards with logs/metrics consumption and analysis ● Worked together with Cloudera team to solve and report issues ● Worked with COE team to develop application deployment solutions on ● Docker/Kubernetes ● Built complex NLP (voice response, audio, txt files) ● Delivery included: Spark, Scala, Hive, Impala, Docker, Kubernetes,Hadoop,HDFS, Pyspark, Oracle, SQL Server, EMR, Redshift, S3, Glue, CloudFormation, Cassandra, Sqoop, Kafka, SparkStreaming, and Nifi, Azure stack (Azure datafactory, azure databricks, cosmodb, storage account, azure datalake,Keyvault, Service Principal(app registration),azure functions, azure sql server)

Azure
Oracle
Spark
SQL Server
icon
Big Data Analyst

Stefanini Perú

Client: Banking(Scotiabank)

● Developed and maintained ETL flows to load data into the warehouse from the systems collecting data

Responsibilities & achievements

● Developed, maintained and reported data warehouse ● Performed analysis and modeling for different areas (Portfolio management, credit policies, credit loss models, among others) ● Defined and managed views in the warehouse to meet the requirements of data scientists using platforms like Spark Databricks and business users using visualization tools like Tableau ● Developed and maintained data stream processing workflows for device event data, supporting the needs of business users for up-to-date information and customer-facing services ● Built dashboards for different areas (Marketing, Sales, among others) ● Delivery included: SQL Server, Oracle, Pyspark, S3, Redshift, CloudFormation ,Glue, QuickSight,Hadoop,HDFS, Power Bi, and Qliksense, Airflow,git, Azure(Azure datalake gen2,DataFactory, StorageAccount,Databricks)

Azure
Oracle
SQL Server
Power BI
icon
Software Engineer III

Avantica

Client: Appetize

● Coded for back-end, microservices implementation, batch processes and unit tests

Responsibilities & achievements

● Developed APIs and deployed them ● Reviewed and made unit tests to solutions ● Developed batch processes to load data ● Interacted with partners and customers to help define roadmap and shape the technology ● Delivery included: Oracle, SQL Server, NodeJ.js, Python,pandas, Flask, and Docker, GCP( Apache Beam, Google Storage, Google Functions, Cloud Scheduler,BigQuery, Dataflow, Google Analytics)

Flask
Node.js
Oracle
Python
SQL Server
icon
Data Analyst

Global Sales Solutions

Client: Banking(Credit Bank of Peru, BBVA)

● Interpreted data and analyzed results using statistical techniques and provided ongoing reports

Responsibilities & achievements

● Developed and maintained SQL reports ● Identified, researched, and solved reporting, databases or related applications’ performance problems with internal and external customers ● Documented relevant system parameters, changes, programs and procedures ● Performed data conversions and analysis (ETL, DW) ● Delivery included: SQLServer, SRSS, SSIS, and Power Bi,python,pandas,flask, Salesforce marketing cloud,salesforce sales cloud

Flask
Python
SQL Server
pandas
icon
Software Engineer

Reach IT

Client: British Telecom Spain

● Built APIs and worked together with IT operations in deployment solutions

Responsibilities & achievements

● Stored procedures optimization ● Was responsible for the database modeling ● Built ETLs flows to load and transform data ● Solved issues in existing projects and made improvements ● Delivery included: Node.js, Pentaho, SQL Server, and Oracle, Heroku, GCP (Apache Beam, Spark, Google Storage, Google Functions, Cloud Scheduler, BigQuery, Dataflow),Python, flask, pandas, flask.

Flask
Python
pandas

Education

Higher education in Computer Science

Agency

Software development agency #3757

10-50

GMT-5

Lima, Peru

Core Expertise

Agile
Amazon EC2
Amazon S3
AngularJS
AWS
Azure
C#
Django
Elixir
ETL
Express.js
Flask
Google Analytics
Groovy
Hibernate
HTML5
Ionic
Java
JavaScript
jQuery
Kotlin
Kubernetes
Microsoft
Microsoft Dynamics CRM
MongoDB
.NET
Node.js
PHP
PostgreSQL
Python
QlikView
React.js
React Native
Ruby on Rails
Scala
Selenium
Spark
Spring
SQL
SQL Server
SSIS
Tableau
TypeScript
WordPress
Xamarin
Apache Tomcat
Bootstrap
CSS3
Git
Go
Golang
HTML
iOS
Mocha
Oracle database
Pentaho
Project Scheduling
Scrum
SQL Azure
SQL Programming
Unit Testing
Web Services
Sketch
User Experience Design
Angular 2x
Postman
Project management
Docker
DynamoDB
MariaDB
SQL query
InVision
Redux
Project Manager
Scrum Master
Maven
Spring Boot
Illustrator
Photoshop
Jest
Enzyme
Hadoop
Flutter
.NET Core
Figma
AWS Lambda
Firebase
Next.js
SEO
Power BI
AWS Glue
Pyspark
.NET Framework
Snowflake
SAP HANA

Industries

Architecture & Design, E-Commerce & Retail, Information services & Technologies, Construction & Real estate, Data Science & Machine Learning, Branding, design, web development

Want to hire this engineer?

Check if Andy is available