Related Issues

This section is mostly about non-Rails and non-Ruby issues, but they're closely related.

See part 2 for solution of troubles described in this section

If you have an MS SQL Server database which must be migrated to MySQL, chances are good that you'll run into some nasty character encoding issues. MS SQL Server usually encodes all textual data in latin1 (ISO 8859-1, see http://en.wikipedia.org/wiki/ISO-8859-1). You can, of course, use this encoding schema for your MySQL database too.

Rails, however, practically demands that you use utf-8. This is because all ajax requests are encoded in utf-8. I have actually experienced this myself, and this link confirms it: http://dema.ruby.com.br/articles/2005/07/22/playing-nice-with-ajax-and-iso-8859-1

The problem is general to all ajax related web development: it's not Rails' fault, it's XmlHttpRequest's fault. Anyway, the solution presented on the latter website did not work for me. Maybe I did not have the Ruby iconv library installed, but it felt like a hack anyway.

So, time to get the database into utf8 encoding.

This part is easy. Just follow the steps described elsewhere in this document: i.e. use a tool such as Access2MySQL, which allows you to specify the character encoding for the destination MySQL database.

This should be all right? Yes, unless you want migrate you MySQL database to another MySQL database which resides on a Linux machine. The only way to do this, as far as I know, is to do a database dump (using a tool such as phpAdmin); uploading the .sql file to the Linux server; and doing a command like:

mysql -uUSER -pPASSWORD DATABASE_NAME < FILENAME.sql

Of course, this all works. But the Unix/Linux file system is used to pipe the file data to the mysql program. And the file system, at least on my machine, does not seem to know utf8…

Probable solution: use a text editor to convert the database dump file to latin1. Do a search and replace for each table create statement.

CREATE TABLE `account_stats` (
  `id` int(11) NOT NULL auto_increment,
  `status` char(50) default NULL,
  KEY `Auto_Increment_Key` (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=4 ;

Create a new database on the linux machine, using this statement:

CREATE DATABASE `database_name` DEFAULT CHARACTER SET latin1;

Once you've got all data transferred to the MySQL database in latin1 encoding, it's time change everything back to utf8. But this seems to involve using iconv… See http://www.loudthinking.com/arc/000415.html

If I use my text editor (pspad) to save a file in utf8 and then upload it to my fedora linux box, all special characters get messed up.

The locale for my linux box was utf-8 all the time!

[root@1038 mysql]# locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

I have not idea what could be the cause of my MySQL troubles… I think using the iconv library to remedy the ajax situation might be the best idea after all…

After some experiments, I have concluded that the database dump file is actually encoded in UTF8 now. Also, the database itself contains UTF8 text fields - if I do not start my terminal in UTF8 mode, it shows garbage when I look at the UTF8-encoded special characters. You can test this using the mysql utility to perform database queries.

(Use echo -e '\e%G' on the prompt to get your KDE or GNOME terminal into utf-8 mode).

If I use webmin to look at the database, it shows garbage too (just like in the Rails app), but that is almost certainly related to the fact that Webmin html pages contain this meta tag:

<meta http-equiv="Content-Type" content="text/html; Charset=iso-8859-1">

So, in terms of MySQL, the Webmin page encodes for latin1 instead of utf8. To fully understand this, just make an xhtml file, add some special characters encoded in utf-8 (and save the file in utf-8 format), but then use the meta tag mentioned above and see how the special characters are messed up in every single browser.

The question remains: why does Ruby on Rails have trouble showing the correct characters, even if database.yml contains this instruction:

development:
  adapter: mysql
  database: database_name
  encoding: utf8    
  username: username
  password: password
  host: localhost

Also, my application.rhtml contains this header:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html
     PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
     "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
  <head>
    <meta http-equiv="content-type" content="text/html; charset=UTF-8">

Finally, I have also included this in my environment.rb file:

# Make Ruby understand UTF-8, see http://manuals.rubyonrails.com/read/chapter/105
# (Added 20060621, Onno Schuit, www.solin.nl -- I actually don't think this should # be necessary)
$KCODE = 'u'
require 'jcode'

Conclusion: somewhere along the route from database to rails to html, the utf-8 encoding gets lost. Most likely upon retrieval of the text fields. Are there any other Rails configuration files or options that I am not aware of?

It's unlikely, as the phpMyAdmin utility messes up the special characters as well.

I finally found the solution here:

http://bruno.vernay.free.fr/wp/?cat=3

This is what you've got to do:

[root@1038 mysql]# mysql -uUserName -pPassword --default-character-set=utf8 database_name < database_dump.sql

Take a look at the MySQL manual as well:

http://dev.mysql.com/doc/refman/4.1/en/charset-connection.html

This explains a lot about the way MySQL interacts with clients with regard to character sets.

The MySQL server now sends the correct characters to the client. And that's really all it takes.

And if it doesn't, add encoding: utf8 to your database.yml file.

Tinymce is a nice little online texteditor. The project's website, http://tinymce.moxiecode.com/example_full.php?example=true, does not really have any tutorials available, however.

Here's how you integrate Tinymce with Rails:

http://www.busyashell.com/blog/articles/2006/05/03/tiny-mce-living-in-your-rails-forms

And in this blog I have added some comments about using Ajax in combination with TinyMCE. Don't forget to include the following code at the end of your partial:

<script type="text/javascript" >
  tinyMCE.idCounter=0;
  tinyMCE.execCommand('mceAddControl', true, 'id_of_your_textarea');  
</script> 

In Tinymce you can display your own css classes in a combobox. Here's a configuration example:

    <script type="text/javascript">
 
      tinyMCE.init({      
        mode : "textareas",
        language : "nl",
        editor_selector : "tiny_mce",        
        theme : "advanced",        
        theme_advanced_buttons1 : "bold,underline, bullist,numlist,styleselect",        
        theme_advanced_buttons2 : " ",        
        theme_advanced_buttons3 : " ",
        auto_reset_designmode : true,
        content_css : "/stylesheets/in_editor.css"                     
      });
 
    </script> 

Put this code in your views/layouts/application.rhtml file, or another file in views/layouts if you want to override default behavior or settings.

The setting styleselect displays the combobox. The property content_css is used to specify the contents of the combobox, as derived from a css file.

The “dollar” function does not work as a shortcut for “document.getElementById” if you do not quote the id string!

E.g. :condition ⇒ ”!($(user_protect_startenddates).checked)” results in the MS IE error “user_protect_startenddates is not defined”.

Fix: rewrite as :condition ⇒ ”!($('user_protect_startenddates').checked)”.

The drag 'n drop libraries work like a charm. There's one issue though: your draggables stop being draggable after you have dropped them.

Fix: put the droppable divs before the draggable divs. This is necessary even if you position them with position:absolute in your css.

As stated on its homepage, “RMagick is an interface between the Ruby programming language and ImageMagick/ GraphicsMagick.”

You can attempt to install the prerequisite library GraphicsMagick with apt-get, but most of the time you'll get stuck with an out-of-date version (i.e. RMagick requires a newer version).

Install RMagick itself by following the instructions on this part of the homepage: Installing RMagick 2 on Linux.

Be especially sure to follow the bit of advice that says: do an irb session to test the installation.

$ irb -rubygems -r RMagick
irb(main):001:0> puts Magick::Long_version
This is RMagick 2.5.2 ($Date: 2008/08/24 21:17:09 $) Copyright (C) 2008 by Timothy P. Hunter
Built with ImageMagick 6.4.3 2008-08-26 Q8 http://www.imagemagick.org
Built for ruby 1.8.7
Web page: http://rmagick.rubyforge.org
Email: rmagick@rubyforge.org
=> nil

You may get a 'directory not found' type of error:

onno@mysystem:/home/familymemorygames/log# irb -rubygems -r RMagick
/opt/ruby-enterprise-1.8.6-20081215/lib/ruby/gems/1.8/gems/rmagick-2.8.0/lib/RMagick2.so: libMagickCore.so.1: cannot open shared object file: No such file or directory - /opt/ruby-enterprise-1.8.6-20081215/lib/ruby/gems/1.8/gems/rmagick-2.8.0/lib/RMagick2.so (LoadError)

The solution: use locate to find libMagickCore.so.1 and be sure to include its path in /etc/ld.so.conf (the mini text editor nano is an excellent tool for this). Here's what I had to include on an Ubuntu 6.06 system:

# cat /etc/ld.so.conf
/usr/local/lib/

After you've added the path to the GraphicsMagick library (i.e. libMagickCore.so.1), be sure to issue the command ldconfig. ldconfig creates the necessary links and cache to the most recent shared libraries found in the directories specified in the file /etc/ld.so.conf.

Here we present a completely unobtrusive way to use the jQuery datepicker plugin. Whenever javascript is disabled, the user is using the default Rails date_select select boxes. If, however, javascript is enabled, the magnificent jQuery datepicker is shown, and the Rails select boxes are completely invisible. Of course, they're still there in the background, ready to submit the parsed date to the server - preventing any ambiguities caused by user errors or confusion over date formats.

As an aside: if you're wondering how to get to the actual html dom elements when you've use jQuery's $ function:

var first_header_as_dom_element = $('div.headers').get(0);

Now let's get on with the datepicker code:

# application.html.erb

<%= javascript_include_tag "http://ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.min.js", "http://ajax.googleapis.com/ajax/libs/jqueryui/1.8.4/jquery-ui.min.js", "jquery.ui.datepicker-nl.js", "jquery.rails.js" %>

# _form.html.erb file (a view)

  <p>
    <%= f.label :birthday %>
    <%= f.date_select :birthday, {}, :class => "date_selector"  %><%= f.text_field :birthday, :class => "ui-datepicker" %>
  </p> 

# In your stylesheet

input.ui-datepicker {
  width: 100px;
  display: none;
}

# application.js
$(document).ready(function(){
  // Store selected date in Rails date selector fields
  $('input.ui-datepicker').datepicker({
    onClose: function(dateText, inst) {
      var objDate = $(this).datepicker('getDate');
      $('#' + this.id + '_1i').val( (objDate) ? objDate.getFullYear() : '');
      $('#' + this.id + '_2i').val( (objDate) ? objDate.getMonth() + 1 : '');
      $('#' + this.id + '_3i').val( (objDate) ? objDate.getDate() : '');
    }
  });
  
  // Use Rails date selector fields to parse date into proper format for datepicker input field
  $.each($('input.ui-datepicker'), function(index, element) {
    if (element.value != '') {
      var objDate = new Date;
      objDate.setFullYear( $('#' + element.id + '_1i').val() );
      objDate.setMonth( $('#' + element.id + '_2i').val() -1 );
      objDate.setDate( $('#' + element.id + '_3i').val() );
      $(element).datepicker('setDate', objDate);
    }
    element.style.display = 'inline';
    set_date_selector_display(element.id, 'none')
  });
  
  // use submit callback handler of your validations script if necessary
  $('form').submit(destroy_datepicker_fields);
  
  $.datepicker.setDefaults($.datepicker.regional['nl']);
});


function set_date_selector_display(datepicker_id, value) {
  for(n=1;n<=3;n++) {
    $('#' + datepicker_id + '_' + n + 'i')[0].style.display = value;
    $('#' + datepicker_id + '_' + n + 'i')[0].style.visibility = 'hidden';
    $('#' + datepicker_id + '_' + n + 'i')[0].style.width = '0px';
  }
}


function destroy_datepicker_fields(form) {
  // N.B.: Destroying the datepicker field does not combine well with back 
  // button - history.back().
  $.each($('input.ui-datepicker'), function(index, element) {    
    set_date_selector_display(element.id, 'inline');
    $(element).remove();
  });
  /* Uncomment following line when using submitHandler for
   * client_side_validations plugin */
  // form.submit();
  return true;  
}