Start Now Login

intermix Blog

Best practices and lessons learned for cloud ETL and data engineering.

Crowdsourcing Weather Data With Amazon Redshift

3 min READ
August 6th 2018
Have you ever tried setting up a personal weather station?  Collecting digital weather data in your backyard or on your rooftop has recently become an easy thing to do. Sharing it is easy also: the Citizen Weather Observer Program (CWOP) will transmit your backyard data to NOAA to help with forecasts.  All you have to […]
Lucy Hancock Lucy Hancock

Building a Better Data Pipeline - The Importance of Being Idempotent

4 min READ
July 30th 2018
Introduction At a glance, batch data processing seems simple. Pull data from a source, apply some business logic to it, and load it for later use. When done well, automating these jobs is a huge win. It saves time and empowers decision-makers with fresh and accurate data. But this kind of ETL (Extract, Transform, and […]
Cody Hanson Cody Hanson

Improve Amazon Redshift COPY performance:  Don’t ANALYZE on every COPY

2 min READ
July 24th 2018
Introduction One of the core challenges of using any data warehouse is the process of moving data to a place where the data can be queried. Amazon Redshift provides two methods to access data: 1- copy data into Redshift local storage by using the COPY command 2- use Amazon Redshift Spectrum to query S3 data […]
Paul Lappas Paul Lappas

Zero Downtime Elasticsearch Migrations

6 min READ
July 12th 2018
Introduction At intermix.io, Elasticsearch is a final destination for data that is processed through our data pipeline. Data gets loaded from Amazon Redshift into Elasticsearch via an indexing job. Elasticsearch data then gets served to the intermix.io dashboard to data engineers, giving them a view of the performance and health of their own data pipelines. […]
Stefan Gromoll Stefan Gromoll

3 Steps for Fixing Slow Looker Dashboards with Amazon Redshift

7 min READ
July 10th 2018
Looker is a powerful tool for self-service analytics. A lot of of companies use Looker on top of Amazon Redshift for business intelligence. It helps companies derive value from their data by making it it easy to create custom reports and dashboards. “Slow Looker dashboards” is one of the most frequent issues we hear with […]
Lars Kamp Lars Kamp
1 2 3 4 5 6 7