Thursday, December 6, 2007

Getting session data in irb

The irb love continues.
This has been blogged before but it's just worthy of being repeated. I needed to check whether something had been stored into the session for the last (failed!) request. Easy. Check mysql for the last session record updated in the sessions table. Copy the session id. Pop open script/console and
session = CGI::Session::ActiveRecordStore::Session.find_by_session_id("8346c9dc0df")
where the session id is that mess at the end. Now:
>> session.data
=> {:user=>55, :user_login=>"JaneDoe", :skin=>"europe", :traveler_login=>"JaneDoe", "flash"=>{}, :traveler=>55}
>>
It's live so you can make changes on the fly.

Use irb when rails is too heavy

The user_pictures table has multiple rows for each thumbnail size we are using. We are using attachment_fu for picture uploads. Due to programmer stupidity we only added the other attribute data (like titles, descriptions, etc) to one of the thumbnail rows. How to propagate the existing data to their companion rows after hundreds of pictures had already been uploaded by users?

Tried script/console but the update action triggered attachment_fu to re-get the size of the referenced picture. Soooo sloooooow. In fact, too slow to run practically (more than a couple hours). Also tried mysql but got caught up in attempting an UPDATE with a sub-query, and failed to get the syntax right for an UPDATE with a self-referential quasi-join. Note to self - give this stuff to the dba.

Finally used plain old irb. Opened an irb session and the first line was: require 'config/environment'. This loaded ActiveRecord and all the configurations needed to work with the existing models, but without the cruft of the rest of the rails app. The ruby to update all the rows appropriately was trivial. Took all of 15 seconds to complete.

I love ruby. And rails. But ruby more.

Sunday, December 2, 2007

PNG transparancy in IE5.5/6.0 with no size

You've seen the very handy rails plugins for handling png transparency in IE 5.5 and IE 6.0: by Christopher Petersen and by The Rails Way. Don't forget Sam Stephenson's fast/clever function to get a png's size.

They all fail if you are dynamically generating a png image with transparency on the fly. But we have a fix! First some background.

You'd probably be doing this as a controller action that renders directly. The final line of your action probably looks like this:


render :text => slide_image, :status => 200, :content_type => 'image/png', :layout => false


I never figured out if you needed to actually register the png mime type for rails to handle this or if it's already registered, I simply did it anyway. Put this in your environment.rb:


Mime::Type.register "image/png", :png


I use the excellent methods from Christopher Petersen. Assay Depot has several very useful plugins on their blog - Christopher is the software guru there. So I've built on his work. His, and the other techniques, use Microsoft's AlphaImageLoader as part of a filter transform applied to the image. This is an IE-only style feature and is ignored by all other browsers. For safety, the code sniffs the browser and doesn't use the technique for other browsers. The filter usually displays the transformed image (with the correct alpha transparency) and the original image next to it and uses this to size the transformed image. To prevent this looking silly, Christopher's technique replaces the original image with a 1x1 transparent gif sized with the original image's dimensions. But what if you don't know the original image's dimensions?

There's another way to use Microsoft's AlphaImageLoader that does not require an image file size. The trick is to actually supply the image but use another transform filter to make it transparent, too. It requires a layout be applied to the image - usually width or height or position:absolute - all of which would not work for generalized image placement. However you can use the CONTENTEDITABLE html attribute as well. This attribute was invented by Microsoft and although it's part of the HTML 5 draft spec, no other browser has ever provided support for it. It has a couple drawbacks or side effects:
  • the image becomes part of the tab sequence. Not a big deal for me since I'm using the image in a link anyway.
  • the Web user can "edit" the image. The only real editing you can do is delete or resize the image for that rendering only.
If these are an issue, you might want to substitute a CSS style of postion:absolute instead.

Below is Christopher's method pair updated with code to use this technique if no size is supplied. Enjoy! Many thanks to Assay Depot for the initial work!


def png_tag(source, options = {})
options.symbolize_keys!
options[:src] = image_path(source)
options[:alt] ||= File.basename(options[:src], '.*').split('.').first.capitalize
if options[:size]
options[:width], options[:height] = options[:size].split("x") if options[:size] =~ %r{^\d+x\d+$}
options.delete(:size)
end
if ie? && ie? < source ="~" src =" options[:src]" src="'#{src}'," sizingmethod="'crop');" src="">"
else
content_tag("span",
image_tag(options[:src],
:style=>"filter:progid:DXImageTransform.Microsoft.Alpha(opacity=0)"),
:style=>"filter:progid:DXImageTransform.Microsoft.AlphaImageLoader(src=#{options[:src]}, sizingMethod='crop');",
:CONTENTEDITABLE=>true)
end
else
"<img src="" />"
end
end

# Called by png_tag above.
def ie?
m = /MSIE\s+([0-9, \.]+)/.match(request.user_agent)
unless m.nil?
m[1].to_f
end
end
Ugh. If you can't read that code snippet very well, download it from here. I really must get a real code formatter for this blog.

Tuesday, October 2, 2007

poor man's dashboard

Just when I thought I'd found all the simple stuff in ruby, I found %x.

I needed to display the tail of a log file. Thoughts of opening the file, pushing each line into a fixed length array, popping the last line off, etc filled my head. Duh - just shell out and do a tail.

Here's the one-liner as it appears in a view. It also re-sorts to display latest first:

<p><%= %x(tail -n 30 log/activity_log.log).split('\n').reverse.join('<br/>') %></p>

%x shells out to the os and returns stdout (if any) in a string.

Sunday, September 23, 2007

Updating checkboxes

Ran into the weirdest problem today. I have a couple check-boxes on a form. Using form_for to display the check-box fields for update. Even though the submit action on the form delivers the correct params for the check-box fields, the user.update_attributes(params[:user]) generates a SQL UPDATE statement with the wrong values - but only for the check-boxes. All other fields are fine. Here's two screen shots from the development.log that illustrate it.









To work around it, I just added user.update_attribute(:email_1, params[:user][:email_1]) and saved it.

Weird.

Thursday, September 20, 2007

Setting focus to a form field

Rails doesn't set focus to the first form field automatically so you have to do it yourself with a little unobtrusive javascript. Here's a helper that does it:


def set_form_focus
javascript_tag("Event.observe(window, 'load', function() {
var firstForm = $A(document.getElementsByTagName('form'))[0];
Form.focusFirstElement(firstForm);
});")
end


Uses the Prototype library, so the page must have a javascript_include_tag("prototype") or javascript_include_tag("defaults") or equivalent. The Prototype Form.focusFirstElement requires a form id but my forms often don't have a name or id. So we look for the first form on the page using the DOM function getElementsByTagName. This function returns a node list and it's easier to treat this as an array by passing it to the $A prototype function so that [0] gives the first form. Finally, attach this function to the window load event instead of firing before that. Most browsers would show the focus fine if it fires earlier, but IE ends the whole page to be loaded first.

Place the helper after the element on the page, preferably after the <% end %> tag for the form. One tricky thing: if you use tables to build formatted forms (as I do) make sure the <table> tag is after the <% form_for... or <% form_tag... or <% form_remote_for... tag.

Sometimes I use a formless input field with an associated observer to return results via ajax. To set focus to this field I use this quick helper:


def set_focus(id)
javascript_tag("Event.observe(window, 'load', $('#{id}').focus());")
end


Again, it must be after the specified field on the page and this field must have an id.

Saturday, September 8, 2007

Stylesheets in ActionMailer

I love ActionMailer. When sending html emails you might want to reuse your existing CSS stylesheet(s). There are potential problems with doing this (see CampaignMonitor for a definitive run down on styling for email clients), but you might be clever enough to have a page layout that is directly usable for html emails, too.

Most/all email clients won't import a linked stylesheet like a browser does, instead you have to embed it directly in the email as an inline style.

To embed your styles in the email insert something like this in the <head> of your email template:


<style type="text/css">
<%= render :file=>"../../../public/stylesheets/admin.css", :use_full_path=>true %>
</style>


Then you can go ahead and use the similar layout as your Web pages in your email. As the Campaign Monitor guys will tell you, you want to avoid floated elements and any information in rendered images. This usually means you'll make a table-based layout for the email but you can reuse styling from the Web pages.

Friday, August 17, 2007

EXIF to auto-locate pictures

We allow travelers to upload their personal pictures to replace the stock photos we provide. The traveler will want to add a caption and description then assign the picture to a specific day of the tour. The EXIF information embedded by most modern digital cameras into each picture really helps. By knowing the date/time the picture was taken, we can auto-assign it for the traveler.

Here's how to do it with RMagick:


image = Magick::Image.read('/users/jimjames99/desktop/cat.jpg').first

=> /users/jimjames99/desktop/cat.jpg JPEG 500x295 500x295+0+0 DirectClass 8-bit 132kb

>> image.get_exif_by_entry('DateTime')

=> [["DateTime", "2007:07:31 14:18:15"]]


If there is no date/time encoded into the picture you get this result:


=> [["DateTime", ""]]


There are lots of other EXIF entries that you can query, including make and model of the camera, picture settings, and so on. The specification is here (pdf).

A couple of factors conspire to frustrate this user helper. iPhoto and other picture gallery programs tend to either wipe the EXIF data from pictures or removes it when you export the pictures to upload to another system (like the travel portal). And of course just because the traveler took the picture in Rome doesn't mean that they want to show it on the Rome page. So we have to provide a robust picture assignment functionality. More on that another time.

Thursday, July 26, 2007

Inner_text and HTML entities

Again using the wonderful Hpricot to parse another tour operator Web site. Part of the page text contained Ch& acirc;teau which of course was rendered by the browser as: Ch√Ęteau. The page was declaring Latin-1 character coding, but by using HTML entities, the authors made sure the browser did the right thing.

However, when hpricot was pulling the text using element.inner_text, I kept getting Ch?teau as the result. I guess hpricot is converting to some character set (pure speculation) and failing. Testing in irb, I made sure I'd done a $KCODE='u' to allow the console session to display unicode characters (actually I tried every setting for $KCODE) but it kept producing the Ch?teau.

The solution was to use inner_HTML instead. Returns Ch& acirc;teau which I can deal with.

Update: Hi Andrew,

I don't know if hpricot has changed since, but this is how I deal with this class of problems now. I use a nice plugin called HTMLEntities (Google is your friend)

# Tauck tours.
# Note the biggest problem is that Tauck formats their title line differently for different tours.
# Sometimes it's "Day 2 &endash; Historic Rouen", other times "Day 1: Welcome to Rome".
# Not worth the brain bruising of a regex to fix it. Just plan on modifying source when we work on Tauck.
def self.scrape_4(url)
logger = Logger.new("log/development.log")
coder = HTMLEntities.new
doc = Hpricot(open(url))
encoded_mess = doc/("#paneitinerary")
itinerary_portion = Hpricot(coder.decode(encoded_mess.to_html))
itin = Array.new
day = 1
itinerary_portion.search("span.days").each do |item|
etc...

Monday, June 25, 2007

submit_to_remote

I made a novice mistake today - tried to nest one form inside another. The functionality I was looking for is to create an entity that involves pictures. On the page where the entity gets created, I wanted to prefetch the picture specified in a text field via ajax. My initial solution was to make a standard form using rails form_for and link_to_remote. But for the life of me I could not get the link_to_remote to put the value of the text field into params. I don't think it's possible. So I just wrote my own link using prototype's Ajax.updater. The called action rendered an RJS template that made the image tag to display the picture.

But I got the weirdest result: the javascript was being rendered into the browser, instead of the result of the javascript. Thank goodness Brian put his finger on it: wrong mime type being supplied in the response. Why did that happen? Because I mixed my own ajax scripting with the rails scripting in RJS. So I converted everything to my own ajax calls. Worked fine but I'd be stuck with maintaining my own javascript for the rest of the application. Yuk.

So I backed out the ajax script and re-thought the application. My next attempt was to insert a remote_form_for. Which of course died because it creates a form nested inside my main form. HTML does not permit nested forms. Turns out rails has a helper exactly for this: submit_to_remote. It not only makes the ajax call without creating a form, it also puts each of the form fields into params for you. Problem solved.

Saturday, June 16, 2007

Quotes

Note to self. I love this style for blockquotes and must find a use for it in the journal feature of the travel portal. It's a Wordpress theme used by Noah Winecoff.

Friday, June 8, 2007

More than development.log but less than breakpointing

Ran into a problem today that wouldn't reveal itself in the development.log but didn't seem worthy of firing up breakpointing. I had added some more parsing features to the tour operator site scraping tool and I got odd errors on a few pages. The errors didn't show up right away, they were toward the end of the page scraping process. Quick background: The pages consist of tour itineraries listing the activities of each day throughout the tour. Fortunately each day has the same markup. Using hpricot the tool first collects the contents of tags that identify the day of the tour, then cycles back to collect the city each day is at, then cycles back to get the description of each day's activities, and so on. Many passes through the same datastream. At the end, the tool splices all the collections together to form records corresponding to each day's data points.

The error was appearing toward the end of the process. Breakpointing would have had to jump in and out of irb many times before the error appeared.

I had forgotten about logger - the ruby facility to generate your own log messages. So easy to use:
logger.info("processing day: #{itin_day.inner_html[4,3]}")

writes "processing day: 4" to development.log. Half a dozen of these scattered through the tool provided enough information in the log to see exactly where the problem was.

Thanks logger!

Monday, May 28, 2007

hpricot and .NET sites

Need to scrape tour operator sites to extract itinerary information. Some of them provide a Web service, but since they're mostly SOAP, I'd rather just scrape the site to get the data. That way, we'll have a similar tool to use for all tour operators.

Using _why's hpricot tool for the scraping. Had great success with it scraping blogger sites in the past. However, I got this error message on opening the url:
Hpricot::ParseError (ran out of buffer space on element <input>...)
Tried other pages on the same site and other pages just to make sure I hadn't messed up the open method. No problems. Glanced at the page source in Firefox's source display - no evident errors.

First thought was to run the tour operator page through a validator to see if there were missing tag closures or weird tags. Found lots of errors (56) but none seemed like they would cause overrun of buffer space on initial parsing. So I pasted the page into textmate to start removing each error one at a time to identify the culprit.
Whoa! That hidden input tag used by .NET to track state - viewstate - is huge! No wonder it blows the attribute buffer. It wasn't evident in the Firefox view without word wrap.
Sure enough _why has provided a method to increase this if you run into .NET pages like this. Simply increase the buffer size before you try to open such a url:

Hpricot.buffer_size = 262144
doc = Hpricot(open("http://globusjourneys.com/product.aspx?content=itin&trip=7ZJ"))

Thanks _why!

Friday, May 11, 2007

Context Sensitive Help in Rails

I promised some friends I'd do a writeup on context sensitive help and it's led me to think that maybe a plugin will be in order. Walk with me first, through this, and we'll decide together if it's worth a plugin.

Step 1:

You're going to need a Rails application.

>rails cshelp -d sqlite3

Let's generate a few controllers so we can have some context in which to offer help:

>ruby script/generate controller welcome index
>ruby script/generate controller register index

Because I like to give the kids up in Marketing the ability to manage their page titles and SEO, let's go ahead and give them all the content management pieces too:

ruby script/generate scaffold_resource content_page controller:string action:string windowtitle:string keywords:string description:string pagetitle:string heading:string subheading:string body:text footer:text

At this point we have a pretty basic structured content form for Marketing. If we fix up our layout to have these fields, we can spend our time coding and let the interns write the pretty words.
In your application.rb, you'll need this :

# call get_content from your controller's action if you want to grab
# content from the content database and display it on an existing
# form or page
def get_content()
intended_path = request.path.split('/')
# code below works for script/server but code above works for tests...
# intended_path = @request.path_parameters['path']
intended_path.delete('')
intended_path[1] = 'index' if intended_path.nitems == 1
@content_page = ContentPage.find_by_controller_and_action(intended_path[0],intended_path[1])
end

In your application_helper.rb, put this little method:

def get_content_value(field, default)
# needs some error checking
return @content_page.send(field) if @content_page
default
end


If there's content for the calling page it will be returned. If not the default value will be returned.

Now in your layout (mine's application.rhtml) take out the fixed SEO fields, and replace with ERB-ized ones:

<meta name="keywords" content="<%= get_content_value("keywords","") -%>"/>

<meta name="description" content="<%= get_content_value("description","") -%>"/>
<title><%= get_content_value("pagetitle","My Default Page Title") -%>

And the meat of the page:

<%= get_content_value("body","") -%>
<%= yield %>

Let's check it out and see if it works!

Migrate (rake db:migrate), fire up a mongrel (ruby script/server), and keeping in mind the two controllers we created earlier, create some content. Navigate to /content_pages/ and make one record for controller "welcome", action "index" and one for controller "register", action "index". Fill in all the fields, we will use them later.

We'll assume that you're marketing department is smart enough to make content for all of your controllers and actions. Now we can make a before_filter in application.rb to get the content for each page:

before_filter :get_content

The astute among you will notice that your page title is already being set to "My Default Page Title". That's because there is no ContentPage model for the "content_page" controller and "index" method. Extra credit: enter some content for your content CRUD screens!

After you've entered content, navigate to the /welcome/ controller and see what happens. Mine says:
Wouldn't it be nice if everybody had a marketing department that created content?
Welcome#index

Find me in app/views/welcome/index.rhtml

Toes, toes and more toes in the footer.
Notice that I didn't even touch the index.rhtml file in the /app/views/welcome/ folder. The default "Welcome#index" is still there. But the framework we've just written wrapped the body, SEO fields, page title and footer around that default content.

Since this is going just the way we planned, let's put our page level help in there now.

ruby script/generate migration add_help
Fill it in:

class AddHelp <>
Add the new help field to your content_page edit view:

<p>
<b>Help</b><br />
<%= f.text_area :help %>
</p>

Repeat as needed, adding it to the show, index and new views.
Migrate up (rake db:migrate) to add the fields to the database.

Here's the fun part. Add the following to your CSS file:
#help {
display: none;
}

and in your application layout, make some room for help. Right above where you're grabbing the content for the body, put this:

<div id="help"><%= get_content_value("help","sorry charlie, no help for you!")%></div>

Now grab the Java script from this page and insert it in a script block in your application layout. Just get the hide & show functions at the top of the page.

Now add a link to hide or show the help:

<%= link_to "Help?", "#", :onclick=>"new Effect.toggle('help','blind'); return false" %>

Now you have controller and action level content and help for your Rails application. Is it worth a plugin? You tell us in the comments.

In the credit where credit is due department, we got the concept and most of the code for the get_content() method from Chad at Bear Den Designs.

To kick this up a notch, Chad also suggests a method_missing on the application controller, so that you can create content pages for controller/action routes that don't exist. We'll write that up another time.