praktikantenblog

I started to add some rss feeds to my thunderbird, with a focus on ruby on rails blogs to keep in touch with. Currently the list is something like 10 blogs long - i’m sure there are many more, and i will report back about the quality of results plopping into my mailbox.

For now i upload a opml-file that can be imported at least into thunderbird. Please feedback if it’s possible / not possible to import it with other readers.

Well, some good links to start could be Mechanize, and Hpricot

Here’s an example on how to use them, thats does only very basically scrape out the links of a search on ‘hpricot’ on the search engine ‘forestle’.

Let’s have a look:

# Released under Creative Commons Attribution-Noncommercial-Share Alike 3.0
require 'rubygems'
require 'open-uri'
require 'hpricot'
require 'mechanize'

agent = Mechanize.new

@page =  agent.get('http://de.forestle.org/search.php?q=hpricot')

doc = Hpricot(@page.body)

(doc/"/html/body/div[2]/div[2]/table/tr/td/div/div/a").each do |result|
  puts result.attributes['href']
end

Another example spider i wrote is this one, to scrape out all of my contacts as vcards from my xing account.

require 'rubygems'
require 'mechanize'

agent = Mechanize.new

page = agent.get 'https://www.xing.com/de/'

form = page.forms.last

form.login_user_name = 'andi@...'
form.login_password = 'Password'

page = agent.submit form

# click the MyContacts link
# Todo: this should go easier, huh?
page = page.links.select {|link| link.to_s.match(/My Contacts/)}.first.click

vcards = []
vcards += page.links.select {|link| link.uri.to_s.match(/vcard/)}

# go through pagination
while next_link = page.links.select {|link| link.to_s.match(/Next/)}.first
  page = next_link.click
  vcards += page.links.select {|link| link.uri.to_s.match(/vcard/)}
end

puts vcards.size

vcards.each do |vcard|
  card = vcard.click
  card.save("cards/#{card.filename.gsub('"','')}")
end

Any Questions left? Ask me, or your local scraping guru!

Good Luck!

Well, to avoid people from telling me they do not know what rails 3.0 will be shipped out with, here’s the link to the full release notes:

http://guides.rails.info/3_0_release_notes.html

I allow myself to quote the pretty self confident introduction to their new release:

Rails 3.0 is ponies and rainbows! It’s going to cook you dinner and fold your laundry. You’re going to wonder how life was ever possible before it arrived. It’s the Best Version of Rails We’ve Ever Done!

I don’t want to go in to any detail about rails 3, because it’s pretty much described on the page that you’ll see when you follow the link above. So go ahead :)

While reading an article about how the behavior of Rails Plugins will change with Rails 3.0 , i stumbled upon this new page ’railsplugins.org’ that engine yard has created to keep track about which plugins are compatible with rails 3, with ruby 1.9, if they run with jruby and if they are thread safe.

Though there are many more requirements one could have on rails plugin, like a solid test coverage, and a proper re-usability (e.g. a slightly different approach then the one the plugin was originally designed for, should still be easy to solve with the plugin), it’s a good start, and a nice list of plugins, currently counting 145.

After all, we should somehow always have an more complex overview over all those plugins, and collect all interesting meta-information, and remember at least which plugin we use in which application. A good start could be to extract this plugin list, and store it in some kind of database for its own, to be able to connect it with our own meta-information.

Who needs help on building a custom scraper, to scrape out the plugin list :)?

Rails 3 ist draussen und es gibt viele tolle neue Aenderungen.

Alles ganz toll.

Lies mehr ->

Today i was once again annoyed, by having tests calling steps like...

Then I should see "Thank you for your Message"

This neccessarly causes the step to fail, whenever the locales for this message are changed, and thats definitely not what we want.

Well, so i thought about it for a minute, came up with the logical conclusion of modifying step definitions to process the localization, was to lazy to test it out, complained in my favourite irc channel, and someone sent me this link: http://www.railway.at/articles/2009/09/12/using-cucumber-to-test-a-multilingual-app/

As it was exactly what i wanted, i added those modified steps to a file i called localized_steps.rb.

Two problems coming up:

1. cucumber did not know wether to use the original step from web_steps.rb or the ones in localized_steps.rb

2. i did not yet change all steps to use localized strings, to they would all fail, and i'd prefer doing it step by step, as there is not the time for modifying all the tests now.

Solution: I explicitly choose wethere i use the step with localization or with a plain string.

    # localized_steps.rb


    When /^I press localized "([^\"]*)"$/ do |key|

      click_button(I18n.t(key))

    end

    

    When /^(?:|I )follow localized "([^\"]*)"$/ do |key|

      click_link(I18n.t(key))

    end

    

    When /^(?:|I )follow localized "([^\"]*)" within "([^\"]*)"$/ do |key, parent|

      click_link_within(parent, I18n.t(key))

    end

    

    Then /^I should see localized "([^\"]*)"$/ do |key|

      response.should contain(I18n.t(key))

    end