Recently, I was planning a data extraction strategy from an API and the goal was to schedule the frequency of data extraction to avoid cached responses, but also to be within rate limits. In order to have data for analysis, I have collected API response every minute for 3 hours. It resulted in 185 files (3 full hours plus several minutes more) with a total size of 6.15 MB saved in JSON New Lines format (and compressed with gzip) and 55746 records. As an analysis tool, I used clickhouse-local. This utility helps us to run SQL against local files without setting up a database, creating tables, and loading data.

Recently, at eet.nu we have been working on migrating a Ruby on Rails application from MySQL to PostgreSQL. Depending on the complexity of an app, there might be many caveats during migration. Here are some notes on issues that we solved during the migration. This article assumes that we work with: Ruby on Rails 4; MySQL 5.6; PostgreSQL 9.4. Quoting and Boolean fields MySQL uses backticks (`) for quoting, while in PostgreSQL double quotes (") are being used. MySQL uses TINYINT datatype to represent boolean fields. Therefore, 1 would represent true and 0 would represent false. PostgreSQL has native boolean datatype.