Archive for June, 2009

Enviando correo de manera asíncrona en Rails usando Workling, Workling-Mailer y RabbitMQ

Posted by jcastaneyra in Howto/tutorial, OpenSource, Ruby on June 10, 2009

Enviando correo de manera asíncrona en Rails usando Workling, Workling-Mailer y RabbitMQ (English)

El trabajar con message queues es bastante interesante, ya que podemos mandar procesos al background y que estos sean procesados de manera asíncrona, un ejemplo podría ser el envío de correos, aunque también podría servir para realizar otras tareas, por ejemplo, como el envío de mensajes sms, generación de reportes, generación de pdf’s, etc.

En esta ocasión les quiero presentar como enviar correos de manera asíncrona haciendo una aplicación sencilla haciendo uso de los puglins Workling y workling-mailer y del sistema RabbitMQ, en teoría, con esto se podría ajustar esta solución fácilmente a cualquier otro proceso que se quiera realizar de manera asíncrona.

Aplicación base

Se crea una aplicación básica con restful_authentication, y con la opción de activación.

Lo primero que hacemos es crear la aplicación en rails

rails mail_async_example -d mysql
 
rm public/index.html
 
script/plugin install git://github.com/technoweenie/restful-authentication.git
 
script/generate authenticated user sessions --include-activation

Creamos nuestra base de datos y migramos

rake db:create
 
rake db:migrate

Y hacemos unos pequeños cambios al código de la aplicación para enviar correo según las instrucciones de restful_authentication, así que agregamos lo suguiente:

config/routes.rb

map.activate 'activate/:activation_code', :controller => 'users', :action => 'activate', :activation_code => nil

config/environment.rb

config.active_record.observers = :user_observer

Con esto ya tendríamos nuestra aplicación básica la cuál podríamos probar de inmediato apuntando en el browser a http://localhost:3000/signup

Y agregando algunas cosillas extras, a app/controller/application_controller.rb agregamos la línea:

include AuthenticatedSystem

Esta misma línea que agregamos al Application controller la removemos de app/controller/sessions_controller.rb y de app/controllers/users_controller.rb

Y creamos un nuevo controlador

script/generate controller portal index

Modificamos config/routes.rb para agregar al último como página root

map.root :controller => "portal"

Por último agregamos un layout para nuestra aplicación:

app/views/layouts/application.html.erb

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
       "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
 
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
  <meta http-equiv="content-type" content="text/html;charset=UTF-8" />
  <title>Contracts: <%= controller.action_name %></title>
  <%= stylesheet_link_tag 'scaffold' %>
</head>
<body>
<% if current_user %>
   <%= link_to 'Logout', logout_path %>
<% else %>
   <%= link_to 'Login', login_path %>
   <%= link_to 'Sign up', signup_path %>
<% end %>
 
<% [:error, :warning, :notice].each do |level|
    if flash[level] -%>
      <div class="flash <%= level.to_s -%>"><%= flash[level] -%></div>
  <% end -%>
<% end -%>
 
<%= yield %>
 
</body>
</html>

Instalando amqp, workling y workling_mailer

Ahora bien, antes de instalar Workling, para poder trabajar con RabbitMQ es necesario instalar la gema de amqp, esta gema requiere también de la gema de EventMachine, por lo que al instalar amqp también instalará EventMachine.

gem sources -a http://gems.github.com/
sudo gem install tmm1-amqp --no-rdoc --no-ri

Vemos nuestro listado de gemas

sudo gem list

Ahora si, a instalar workling y workling_mailer

script/plugin install git://github.com/purzelrakete/workling.git
 
script/plugin install git://github.com/langalex/workling_mailer.git

Para poder utilizar workling con RabbitMQ agregamos al final de config/environment.rb

config.after_initialize do
Workling::Remote.invoker = Workling::Remote::Invokers::EventmachineSubscriber
Workling::Remote.dispatcher = Workling::Remote::Runners::ClientRunner.new
Workling::Remote.dispatcher.client = Workling::Clients::AmqpClient.new
end

Posteriormente en app/models/user_mailer.rb añadimos

include AsynchMail

A la fecha de hoy el plugin de workling tiene un pequeño issue, para resolverlo hacemos lo siguiente en el archivo lib/workling/clients/memcache_queue_client.rb y verificar que contiene:

require 'memcache'

Al principio del archivo, si no lo tienes agrégalo.

Como el amqp corre sobre un ciclo de EventMachine es necesario correr nuestra aplicación con Thin, ya que este servidor soporta EventMachine sin ningún problema. Para instalar Thin corremos

sudo gem install thin --no-rdoc --no-ri

Instalando RabbitMQ

Checar la instalación aquí http://www.jcastaneyra.com/2009/06/07/instalando-rabbitmq/

A correr todos

Nos aseguramos de que el RabbitMQ esté corriendo.

Iniciamos el cliente de workling

script/workling_client start

E iniciamos el servidor thin

thin start

Apuntamos nuestro browser a http://localhost:3000 y nos registramos con un usuario, al verificar el log de desarrollo veremos que el correo ahora es enviado después de todas las últimas operaciones registradas en el log. Podemos probar quitando la línea de include AsynchMail del UserMailer y ver lo que hace, o bien haciendo nuevo proceso que sea pesado y que se realice en background.

Referencias:

http://github.com/christospappas/workling/commit/063b9849b32e0c6140a5ecdb254357770d9bd28f
http://github.com/purzelrakete/workling/
http://github.com/langalex/workling_mailer/
http://github.com/tmm1/amqp/
http://github.com/technoweenie/restful-authentication/
http://www.rabbitmq.com
http://code.macournoyer.com/thin/
http://www.jcastaneyra.com/2009/06/07/instalando-rabbitmq/

Sending mails in asynchronous way in Rails using Workling, Workling-Mailer and RabbitMQ (Español)

Working with message queues is so interesting, since we can put processes in the background in order to be processed in asynchronous way, imagine this to send several mails, or maybe for heavy tasks as sms messages, report generation, pdf generation, etc.

So, with this post I want you to show how to send mails in asynchronous way, we are going to make a small application using Workling, Workling-mailer and as message queue RabbitMQ, actually we could adjust this solution to any other problem with async process requirement.

Base application

We create a simple application with restful_authentication and activation option.

First of all we create the rails app

rails mail_async_example -d mysql
 
rm public/index.html
 
script/plugin install git://github.com/technoweenie/restful-authentication.git
 
script/generate authenticated user sessions --include-activation

Now with rake we create and migrate DB

rake db:create
 
rake db:migrate

Following the restful_authentication instructions to send mail we add this:

config/routes.rb

map.activate 'activate/:activation_code', :controller => 'users', :action => 'activate', :activation_code => nil

config/environment.rb

config.active_record.observers = :user_observer

Our base app is working now, so to test it we get the url http://localhost:3000/signup in the browser.

Add this in app/controller/application_controller.rb:

include AuthenticatedSystem

The same line added to Application controller is removed from app/controller/sessions_controller.rb and app/controllers/users_controller.rb

We create a new controller for portal index

script/generate controller portal index

And modify config/routes.rb to add portal as root

map.root :controller => "portal"

And last we create a layout for our application:

app/views/layouts/application.html.erb

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
       "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
 
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
  <meta http-equiv="content-type" content="text/html;charset=UTF-8" />
  <title>Contracts: <%= controller.action_name %></title>
  <%= stylesheet_link_tag 'scaffold' %>
</head>
<body>
<% if current_user %>
   <%= link_to 'Logout', logout_path %>
<% else %>
   <%= link_to 'Login', login_path %>
   <%= link_to 'Sign up', signup_path %>
<% end %>
 
<% [:error, :warning, :notice].each do |level|
    if flash[level] -%>
      <div class="flash <%= level.to_s -%>"><%= flash[level] -%></div>
  <% end -%>
<% end -%>
 
<%= yield %>
 
</body>
</html>

Installing amqp, workling and workling_mailer

Now, before using Workling, we need RabbitMQ and amqp is needed, this gem also requires EventMachine gem, so, both gems are installed when amqp is installed.

gem sources -a http://gems.github.com/
sudo gem install tmm1-amqp --no-rdoc --no-ri

List your installed gems

sudo gem list

Now, install workling and workling_mailer

script/plugin install git://github.com/purzelrakete/workling.git
 
script/plugin install git://github.com/langalex/workling_mailer.git

Add this to config/environment.rb to allow workling to use RabbitMQ

config.after_initialize do
Workling::Remote.invoker = Workling::Remote::Invokers::EventmachineSubscriber
Workling::Remote.dispatcher = Workling::Remote::Runners::ClientRunner.new
Workling::Remote.dispatcher.client = Workling::Clients::AmqpClient.new
end

To app/models/user_mailer.rb add this

include AsynchMail

Until today I’ve seen a little issue with workling, to solve it we do the folloging in this file lib/workling/clients/memcache_queue_client.rb; verify that you have:

require 'memcache'

Add it if you don’t have it.

Amqp runs in an EventMachine cycle, so it is needed to run our app with Thin, since Thin supports EventMachine without any problem. To install Thin run

sudo gem install thin --no-rdoc --no-ri

Installing RabbitMQ

Follow the instructions here http://www.jcastaneyra.com/2009/06/07/instalando-rabbitmq/

Everybody runs

Cool, naw we ensure that RabbitMQ is running.

Start workling client

script/workling_client start

And start Thin server

thin start

Point with your browser to http://localhost:3000 and sign up a new user, then verify your log and will see that email is sent and registered to the end of log after all operations. We can test removing include AsynchMail from UserMailer and see what happens, it would be great if you test some application with a heavy process working in the background.

Links:

http://github.com/christospappas/workling/commit/063b9849b32e0c6140a5ecdb254357770d9bd28f
http://github.com/purzelrakete/workling/
http://github.com/langalex/workling_mailer/
http://github.com/tmm1/amqp/
http://github.com/technoweenie/restful-authentication/
http://www.rabbitmq.com
http://code.macournoyer.com/thin/
http://www.jcastaneyra.com/2009/06/07/instalando-rabbitmq/

, , , , , ,

View Comments

Instalando RabbitMQ

Posted by jcastaneyra in Howto/tutorial, OpenSource on June 7, 2009

Instalando RabbitMQ (English)

RabbitMQ es un sistema de message queue (MQ), el cual provee comunicaciones asíncronas, es decir que el productor y consumidor no tienen la necesidad de interactuar con los mensajes al mismo tiempo, además de que es una implementación del protocolo AMQP (Advanced Message Queuing Protocol), un protocolo para mensajeo de alto rendimiento, y por último decir que RabbitMQ está desarrollado con Erlang, Erlang es un lenguaje de programación funcional.

En Ubuntu 9.04

La instalación de RabbitMQ en Ubuntu 9.04 es tan simple como correr el comando

sudo aptitude install rabbitmq-server

Y con el cual si no has instalado Erlang te mostrará que paquetes tendrá que instalar para que lo tengas también.

jcastaneyra@ubuntu:~/sources$ sudo aptitude install rabbitmq-server
[sudo] password for jcastaneyra:
Reading package lists... Done
Building dependency tree
Reading state information... Done
Reading extended state information
Initializing package states... Done
Writing extended state information... Done
The following NEW packages will be installed:
erlang-base{a} erlang-nox{a} libltdl7{a} odbcinst1debian1{a} unixodbc{a}
The following partially installed packages will be configured:
rabbitmq-server
0 packages upgraded, 5 newly installed, 0 to remove and 8 not upgraded.
Need to get 28.6MB of archives. After unpacking 46.9MB will be used.
Do you want to continue? [Y/n/?]

Una vez instalado ya está corriendo.

En Mac leopard

UPDATE (27/05/2010): Desde hace un rato que ya estoy usando Homebrew para instalar cosas en Mac, ya existe una fórmula para instalar también RabbitMQ y por supuesto Erlang, para más información visitar http://github.com/mxcl/homebrew y seguir las instrucciones para instalar Homebrew. Con RabbitMQ, incluso si se quiere una versión específica de RabbitMQ se puede modificar la fórmula :)

La instalación es a través de los MacPorts, hay que bajar el instalador de la página www.macports.org/install.php, pero para poder compilar es necesario tener XCode instalado. Los MacPorts son instalados en /opt, por lo que hay que asegurarnos que se tengamos agregados los paths necesarios en el profile para ejecutar los comandos de MacPorts.

En mi caso en .profile (pero también puede ser el .bash_login) debe estar algo así (si no está lo agrego):

# MacPorts Installer addition on 2009-02-25_at_15:37:48: adding an appropriate PATH variable for use with MacPorts.
export PATH=/opt/local/bin:/opt/local/sbin:$PATH
# Finished adapting your PATH environment variable for use with MacPorts.
 
# MacPorts Installer addition on 2009-02-25_at_15:37:48: adding an appropriate MANPATH variable for use with MacPorts.
export MANPATH=/opt/local/share/man:$MANPATH
# Finished adapting your MANPATH environment variable for use with MacPorts.

Una vez que están los MacPorts hay que instalar primero erlang, así que ejecutamos

sudo port install erlang

Y esperamos un buen rato a que se instale.

Una vez instalado erlang, bajamos la última versión de RabbitMQ

mkdir /tmp/rabbitmq &amp;&amp; cd /tmp/rabbitmq
curl -O http://www.rabbitmq.com/releases/rabbitmq-server/v1.5.5/rabbitmq-server-generic-unix-1.5.5.tar.gz
tar xvzf rabbitmq-server-generic-unix-1.5.5.tar.gz
sudo mv rabbitmq_server-1.5.5/ /opt/local/lib/erlang/lib

Ahora ya lo podemos ejecutar.

sudo /opt/local/lib/erlang/lib/rabbitmq_server-1.5.5/sbin/rabbitmq-server

Recursos

http://www.rabbitmq.com
http://playtype.net/past/2008/10/9/installing_rabbitmq_on_osx/

Installing RabbitMQ (Español)

RabbitMQ is a complete and highly reliable Enterprise Messaging system, it provides asynchronous communications,  meaning that the sender and receiver of the message do not need to interact with the message queue at the same time, also it is a AMQP implementation (Advanced Message Queuing Protocol) a protocol for high performance messaging, and last RabbitMQ is developed with Erlang, a functional programming language.

In Ubuntu 9.04

RabbitMQ instalation in Ubuntu 9.04 is so simple, it is just needed to run a command

sudo aptitude install rabbitmq-server

With this also install all packages needed by rabbitmq, Erlang is installed also.

jcastaneyra@ubuntu:~/sources$ sudo aptitude install rabbitmq-server
[sudo] password for jcastaneyra:
Reading package lists... Done
Building dependency tree
Reading state information... Done
Reading extended state information
Initializing package states... Done
Writing extended state information... Done
The following NEW packages will be installed:
erlang-base{a} erlang-nox{a} libltdl7{a} odbcinst1debian1{a} unixodbc{a}
The following partially installed packages will be configured:
rabbitmq-server
0 packages upgraded, 5 newly installed, 0 to remove and 8 not upgraded.
Need to get 28.6MB of archives. After unpacking 46.9MB will be used.
Do you want to continue? [Y/n/?]

Once the installation is finished RabbitMQ will be running.

In Mac leopard

UPDATE (27/05/2010): I am using Homebrew to install some stuff in Mac from some months ago, Homebrew has a formula to install RabbitMQ and also Erlang, for more info please visit http://github.com/mxcl/homebrew for install instructions. Also if you want to install a different version of RabbitMQ you can do it just editing the formula :)

RabbitMQ installation is done through MacPorts, you need to download the MacPorts installer from www.macports.org/install.php, also XCode is needed. MacPorts are installed in /opt, for that reason we put these paths in the .profile file (could be .bash_login), if you don’t have these paths add them to your profile file:

# MacPorts Installer addition on 2009-02-25_at_15:37:48: adding an appropriate PATH variable for use with MacPorts.
export PATH=/opt/local/bin:/opt/local/sbin:$PATH
# Finished adapting your PATH environment variable for use with MacPorts.
 
# MacPorts Installer addition on 2009-02-25_at_15:37:48: adding an appropriate MANPATH variable for use with MacPorts.
export MANPATH=/opt/local/share/man:$MANPATH
# Finished adapting your MANPATH environment variable for use with MacPorts.

In order to install erlang execute the following command:

sudo port install erlang

You need to wait some time, maybe several minutes :)

Once you have erlang installed, we download the last RabbitMQ version

mkdir /tmp/rabbitmq &amp;&amp; cd /tmp/rabbitmq
curl -O http://www.rabbitmq.com/releases/rabbitmq-server/v1.5.5/rabbitmq-server-generic-unix-1.5.5.tar.gz
tar xvzf rabbitmq-server-generic-unix-1.5.5.tar.gz
sudo mv rabbitmq_server-1.5.5/ /opt/local/lib/erlang/lib

Now we execute rabbitmq-server with this

sudo /opt/local/lib/erlang/lib/rabbitmq_server-1.5.5/sbin/rabbitmq-server

Links

http://www.rabbitmq.com
http://playtype.net/past/2008/10/9/installing_rabbitmq_on_osx/

, , , ,

View Comments