English:
Upper Intermediate
Peru
UTC -05:00
America/Lima
Strong Senior Data Engineer with more than six years of experience.
Strong Senior Data Engineer with more than six years of experience. I have a diverse background where he has established Data architecture solutions and he’s also led teams using agile methodologies. Strong experienced in relational and non-relational databases, Scala, Python, Spark, and AWS Stack (EMR, Redshift, S3, CloudFormation),GCP, and Azure amongst others for ETL and data pipeline-related work, and distributed file Systems(HADOOP)
Want to hire this engineer?
Check if Andy is availableExpertise
Years of commercial development experience
6 years of experience
Core technologies
Project Highlights
NTT DATA
Built automations and extract data from (salesforce marketing cloud and salesforce sales cloud)
Responsibilities & achievements
● Extract data from third party API´S ● Build pipelines to merged data in the data architecture ● Build dashboards and reports ● Apply improvements to the existing pipelines ● Develop end to end pipelines ● Pull data from different sources(sql, file systems, ftps,etc) ● Delivery included: Google Storage, Airflow, Google Functions, Cloud Scheduler , Python , Dataproc , Dataflow , Cloud composer , Salesforce Api, Snowflake, MongoDB, Kafka, Pyspark, Sql server, Tableau
Spark Digital
● Built automations and extract data from (salesforce marketing cloud and salesforce sales cloud)
Responsibilities & achievements
● Extract data from third party API´S ● Build pipelines to merged data in the data architecture ● Build dashboards and reports ● Apply improvements to the existing pipelines ● Develop end to end pipelines ● Pull data from different sources(sql, file systems, ftps,etc) ● Delivery included: Google Storage, Airflow, Google Functions, Cloud Scheduler, Big query, Python , Dataproc , Dataflow , Cloud composer , Salesforce Api
Everis Peru
Banking(Credit Bank of Peru,Pichincha,Interbank), Assurance(La positiva),Telco(Telefonica)
Lead and built applications with big data technologies within data lake architecture
Responsibilities & achievements
● Provided technical direction and leadership to project teams ● Worked with COE (Center Of Excellence) - BCP in DevOps, DataOps, Machine Learning and Big Data projects ● Created and led POCs to stay up-to-date with advances in data persistence and big data technologies and run experiments/POCs(aws,azure,gcp) ● Build computer visions with security cameras and getting information from sensors ● Built Dashboards with kpi´s ● Designed and proposed architectures to new jobs(aws,azure,gcp) ● Created task to execute sql queries ● Made troubleshooting to existing jobs ● Migrated business logics from sql to pyspark ● Created ETL flows into SFMC ● Extracted information from API´s (WeAreHearken) ● Built complex data pipelines for data processing into datalake architectures(spark) ● Built data pipelines to extract information from different sources and merge the data into same architecture(datalake) ● Deploy virtual environments ● Delivery included: Oracle, SQL Server, Pyspark, Scala, Spark,Hadoop, Hdfs, Hive, Impala, Cassandra, AWS Stack(EMR, Redshift, S3, CloudFormation), ElasticSearch, Kafka, SparkStreaming, Sqoop, and Nifi,OpenCV,NLP,OpenFace,Tensorflow,Salesforce marketing cloud, Salesforce Sales, Airflow, GCP stack(Google Functions, Cloud Scheduler, Dataflow.Google Data Studio, Terraform, BigQuery, Google Storage), Azure(Azure datafactory, azure databricks,cosmodb,storage account, azure datalake,Keyvault, Service Principal(app registration),azure functions, azure sql server,Azure devops,azure synapse analytics, power apps, power automate)
Fit Big Data
Client: Banking(Goldman Sachs)
● Developed applications for batch processing
Responsibilities & achievements
● Developed applications for batch processing with Scala and Spark ● Optimized and applied improvements to existing applications ● Built pipeline for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and big data technologies ● Built analytics tools that utilize the data pipeline to provide actionable insights ● Built ETL for data extraction from the sources ● Built automations and extract data from (salesforce marketing cloud and salesforce ● sales cloud) ● Built and improve accuracy machine learning models (Linear Regressions, decision ● tree), NLP ● Web Scraping and collect data into nosql databases and distributed files ● Delivery included: Qlik Sense, SQL Server, Oracle, Nifi, Spark, Scala, Pyspark, PysparkML,S3, EMR, Athena, Redshift, QuickSight, Kafka, SparkStreaming, Hadoop,Hdfs,Sqoop, Salesforce,Glue, Lambda, marketing cloud, salesforce sales cloud, Airflow, Jenkins,Bitbucket, Fortify
Everis Perú
Client: Banking(Credit Bank of Peru)
● Built batch processing pipelines ● Built dashboards with logs/metrics consumption and analysis
Responsibilities & achievements
● Worked together with Cloudera team to solve and report issues ● Built complex batch and real time jobs ● Extract data from different sources ● Migrating business logics from sql querys to spark Built dashboards with logs/metrics consumption and analysis ● Worked together with Cloudera team to solve and report issues ● Worked with COE team to develop application deployment solutions on ● Docker/Kubernetes ● Built complex NLP (voice response, audio, txt files) ● Delivery included: Spark, Scala, Hive, Impala, Docker, Kubernetes,Hadoop,HDFS, Pyspark, Oracle, SQL Server, EMR, Redshift, S3, Glue, CloudFormation, Cassandra, Sqoop, Kafka, SparkStreaming, and Nifi, Azure stack (Azure datafactory, azure databricks, cosmodb, storage account, azure datalake,Keyvault, Service Principal(app registration),azure functions, azure sql server)
Stefanini Perú
Client: Banking(Scotiabank)
● Developed and maintained ETL flows to load data into the warehouse from the systems collecting data
Responsibilities & achievements
● Developed, maintained and reported data warehouse ● Performed analysis and modeling for different areas (Portfolio management, credit policies, credit loss models, among others) ● Defined and managed views in the warehouse to meet the requirements of data scientists using platforms like Spark Databricks and business users using visualization tools like Tableau ● Developed and maintained data stream processing workflows for device event data, supporting the needs of business users for up-to-date information and customer-facing services ● Built dashboards for different areas (Marketing, Sales, among others) ● Delivery included: SQL Server, Oracle, Pyspark, S3, Redshift, CloudFormation ,Glue, QuickSight,Hadoop,HDFS, Power Bi, and Qliksense, Airflow,git, Azure(Azure datalake gen2,DataFactory, StorageAccount,Databricks)
Avantica
Client: Appetize
● Coded for back-end, microservices implementation, batch processes and unit tests
Responsibilities & achievements
● Developed APIs and deployed them ● Reviewed and made unit tests to solutions ● Developed batch processes to load data ● Interacted with partners and customers to help define roadmap and shape the technology ● Delivery included: Oracle, SQL Server, NodeJ.js, Python,pandas, Flask, and Docker, GCP( Apache Beam, Google Storage, Google Functions, Cloud Scheduler,BigQuery, Dataflow, Google Analytics)
Global Sales Solutions
Client: Banking(Credit Bank of Peru, BBVA)
● Interpreted data and analyzed results using statistical techniques and provided ongoing reports
Responsibilities & achievements
● Developed and maintained SQL reports ● Identified, researched, and solved reporting, databases or related applications’ performance problems with internal and external customers ● Documented relevant system parameters, changes, programs and procedures ● Performed data conversions and analysis (ETL, DW) ● Delivery included: SQLServer, SRSS, SSIS, and Power Bi,python,pandas,flask, Salesforce marketing cloud,salesforce sales cloud
Reach IT
Client: British Telecom Spain
● Built APIs and worked together with IT operations in deployment solutions
Responsibilities & achievements
● Stored procedures optimization ● Was responsible for the database modeling ● Built ETLs flows to load and transform data ● Solved issues in existing projects and made improvements ● Delivery included: Node.js, Pentaho, SQL Server, and Oracle, Heroku, GCP (Apache Beam, Spark, Google Storage, Google Functions, Cloud Scheduler, BigQuery, Dataflow),Python, flask, pandas, flask.
Education
Higher education in Computer Science
Agency
10-50
GMT-5
Lima, Peru
Core Expertise
Industries
Architecture & Design, E-Commerce & Retail, Information services & Technologies, Construction & Real estate, Data Science & Machine Learning, Branding, design, web development
Want to hire this engineer?
Check if Andy is available