Data Engineering

Created by Jordan Skole, VP of Product Growth at Autobooks
685 People have this badge

What everyone's up to

Wrote a Blog Post
Data engineering
+ 1
Just published a blog post about how to build a reliable data ingestion service using FastAPI, SQLModel and Dramatiq ➡️
Data Collection
Setup database
Google BigQuery
+ 1
Generate BigQuery credentials  To let dbt connect to your warehouse, you'll need generate a keyfile. This is like using a database user name and password with most other data warehouses. (but without filling up manually)

  1. Go to the BigQuery credential wizard. Ensure that your new project is selected in the header bar.
  2. Generate credentials with the following options:
    • Which API are you using? BigQuery API
    • What data will you be accessing? Application data (you'll be creating a service account)
    • Are you planning to use this API with App Engine or Compute Engine? No
    • Service account name: dbt-user
    • Role: BigQuery Job User & BigQuery User
    • Key type: JSON
  3. Download the JSON file and save it in an easy-to-remember spot, with a clear filename (e.g. dbt-user-creds.json)

Started a new contract
Data engineering
Onboarded to a New Codebase
+ 1
Started a new contract with polywork
Sep 09, 2021
Started a new job
Built a data pipeline
Used Google Cloud Platform
+ 1
Over the past ~2 months, I've had an amazing opportunity to work with the brilliant team at Sift. Before joining, I had effectively zero experience using SQL or Python, but have since spent many an hour diving deep into the technology and worked to build out an amazing product.

I want to highlight how amazing it is to be a part of team building something. Building the pipeline to take raw, ugly data and transform it into information that provides meaningful value to the end user has been so much fun, and it's great to see how everything develops as the days go on. 

And that doesn't even get into how everyone at Sift is incredibly talented and smart, and have been incredibly gracious sharing their knowledge and their expertise with me.I've grown tremendously in such a short period of time, and I attribute that to the curious and dedicated culture at Sift. 

If you can't tell, I really love working at Sift and am excited to continue learning and growing with the team :)

Completed a training course
Google BigQuery
Self-Directed Learning
+ 1
Data Science
Learnt Python
Data Analyst
Experiment Design
Data engineering
+ 3
Graduated from the "Data Science Nanodegree" (#DSND) program at #Udacity after 6 months 🔥🔥🔥

- Solving Data Science Problems
- Software Engineering for Data Scientists
- Data Engineering for Data Scientists
- Experiment Design and Recommendations
- Data Science Projects