Loading data in migrations

I found migrations to be very useful tool to maintain data structures in a rails project. It is clean, easy to understand and fast when it comes to recreate a database.

During develoment, I faced several times the same problem: “How to import static data into the database?” Of course, one can create fixture and load it. But sometimes one needs more powerful tool.

So, I tried to import the data in the migrations themselves. Since the data to import was huge (tens of MBs), I did try several methods and approaches. Here is the result.

Fixtures in Migrations

This is method that I found on the web in an article of Adam Christensen Loading Fixtures in a Migration. It shows how to load the fixtures file into database.

Active record way

If your ActiveRecord object already exists, you can use the following code. It is obviously not the way to import thousands of records, but for ten objects it is good enough.

External CSV

If the data came in the CSV format then a fastercsv library could be used.

First of all, the library needs to be installed.

then create the migration

And place the CSV file (example below) into db/data folder.

Embedded CSV

Sometimes it is useful to bundle the data directly with the migration. In this example, the data are provided to the load_articles method from articles_data.

ActiveRecord – native commands

Last, but not least is the method that uses native functions of MySQL database. This could work only if the rails application sits on the same machine as the MySQL.

and the corresponding CSV file

That’s it! Enjoy it.

5 Responses to “Loading data in migrations”

  1. nick_jcn@yahoo.com Says:

    I’m looking for a RoR specialist to help us with a project. If you are interested to talk about it, or have any advise, or know somebody who could help us out or join our team – please let me know.

    Nick Borisenko
    nick_jcn@yahoo.com

  2. Gustavo Says:

    Thank you so much for this information.

    I had to add

    require ‘tempfile’

    to your “ActiveRecord – native commands” example for it to work.

    I added it to the end of my config/environment.rb file.

  3. Roman Mackovcak Says:

    2 Gustavo

    Thanks, fixed.

  4. jan Says:

    So which one did you choose and why? You don’t mention it.
    But thanks for this overview, I was a bit procrastinating researching this, so now I don’t have to;)

  5. Roman Mackovcak Says:

    2 Jan

    Every method has got its own pros and cons. I had a big CSV file, so External CSV and Native commands were my options.