ruby

You are currently browsing articles tagged ruby.

Beware of YAML serialization on Linux without libyaml

November 22, 2011 in programming | Add a comment

The Problem

I noticed some unusual behavior with respect to YAML String serialization between my Linux production system and my Mac OSX development system.

After dumping the production database via pg_dump -O –no-acl mydb | gzip > ~/mydb.sql.gz and then restoring it on my development system via rake db:drop; rake db:create; psql mydb < mydb.sql, I noticed that a particular serialized field in my Rails app that should always be an Array of String objects occasionally contained Integers.

After a little research and experimentation, I discovered that the production Linux system would occasionally omit quotations around Strings containing only numeric digits. I haven’t analyzed the pattern fully, but here are some examples where the YAML serialization did or did not use quotes:

  • “90103″
  • 000080
  • “000071″
  • “000124″
  • “000003″
  • 008397
  • 000408
  • 000009
  • 000188
  • “000021″

Further investigation revealed that the Linux production system was using Syck (a “dated C implementation of YAML 1.0″) and my Mac OSX development system was using psych (a “libyaml wrapper (in Ruby core for 1.9.2)”). libyaml is a “fast C implementation of YAML 1.1. So, either the quotation rules have changed between YAML 1.0 and YAML 1.1, or there is a bug in one of the implementations (likely Syck).

The Solution

The solution for proper “future” behavior is pretty simple. Install libyaml on the Linux system as follows:

wget http://pyyaml.org/download/libyaml/yaml-0.1.4.tar.gz
tar xzf yaml-0.1.4.tar.gz
cd yaml-0.1.4
./configure
make
make install

I think that’s enough, but I went ahead and rebuilt my Ruby 1.9.2 just in case it needed to know about the existence of libyaml at build time.

The solution for converting my database with YAML 1.0 serialization to YAML 1.1 serialization is a bit trickier. Since the “dump” and “load” operations are matched for a particular version of YAML, it seems difficult to load the data using YAML 1.0 (thereby retaining the String type when reading an unquoted 000088) and then dump the data using YAML 1.1 (to get proper quoting of ‘000088′). Further complicating this is the fact that Rails handles the serialization operations automatically.

It does appear possible to dynamically switch between syck and psyck by using the following:

YAML::ENGINE.yamler = 'syck'
YAML::ENGINE.yamler = 'psych'

So, one option is to repeatedly switch to syck, read in data, switch to psych, and then write the data. <sigh>

Update:

It appears that due to the semantics of the Rails serialize function, it’s not enough to just read the model object using syck and then immediately write with psych because that doesn’t appear to be enough to cause the field to be deserialized. I had to refer to the field for each object. This is a pain because it prevents me from doing a generic loop where I can handle all model objects easily w/o reference to their specific fields.

I’ll withhold judgment for a while, but my first inclination is to consider abandoning YAML serialization for something a little more robust and portable.

Update 2:

It appears my welcome from psych is a serious memory leak. I’ve been running long running Ruby/Rails processes for years, and this is the first time I’ve experienced a failure due to an out of memory condition. There are a number of Google hits regarding the issue. After I fix the leak, I’ll begin researching alternatives to YAML serialization in Rails.

Update 3:

The number of bug reports on psych and rubygems I’ve had to wade through recently is amazing. My current solution is to remove the psych system gem and install Ruby 1.9.3p0 which required upgrading Passenger to the latest version from source to get Ruby 1.9.3 compatibility. I still had to track down a few odd errors such as “undefined method `yaml’ for #<Psych::Nodes::Stream:…>” and “invalid date format in specification: “2011-10-02 00:00:00.000000000Z”" – all because I chose to use the default Rails serialization assuming there would be no issues. Lesson learned.

Tags: , ,

Programming Language Popularity – Part Four

September 22, 2011 in programming, software | Add a comment

I compiled some programming language popularity statistics in April 2009, October 2009 and October 2010 . Here’s an update for September 2011:

I made a number of Google searches of the forms below and summed the results (previous posts averaged the results):

"implemented in <language>"
  "written in <language>"

Naturally this is of very limited utility, and the numbers are only useful when comparing relatively within the same search since the number of results Google returns can vary greatly over time.

Language Total Prev. Position Position Delta
C 10,360,000 2 1
PHP 10,351,000 1 -1
C++ 6,495,000 3 0
Python 5,759,000 5 1
C# 5,335,000 4 -1
 
Java 4,890,000 8 2
Perl 3,702,000 6 -1
JavaScript 3,077,000 7 -1
Ruby 1,654,000 9 0
Lisp Family1 1,022,870 11 1
 
FORTRAN 975,600 10 -1
Tcl 594,500 12 0
Lisp 486,000 14 1
Haskell 450,500 16 2
Erlang 419,700 13 -2
 
Lua 367,100 18 2
ML Family2 348,400 17 0
COBOL 308,270 15 -3
Common Lisp 254,900 19 0
OCaml 240,300 21 1
 
Prolog 224,000 20 -1
Scala 203,400 23 1
Scheme 184,700 22 -1
Smalltalk 129,700 24 0
Clojure 84,600 27 2
 
(S)ML3 83,630 25 -1
Forth 69,980 26 -1
Caml 24,470 28 0
Io 17,700 30 1
Arc 12,670 29 -1

1 combines Lisp, Scheme, Common Lisp, Arc & Clojure
2 combines OCaml, (S)ML, Caml
3 summed separate searches for sml and ml

Tags: , , , , , , , , , , , ,

Find Longest Substring Palindrome in Haskell

March 13, 2011 in programming | 7 comments

I stumbled upon a programming challenge a company was using for recruitment purposes and thought I’d create a Haskell solution as a learning exercise. The first problem was to find the longest palindrome embedded in a text string.

The following Haskell solution seems very readable to me, but it’s a naive solution that’s inefficient. It computes an answer on my 2.6 year old Macbook Pro in under 4 seconds, but a 2x increase in text requires a 7x increase in CPU time.

I believe there are algorithms to find the longest embedded palindrome in linear time, so I may post a refinement later.

-- Find the longest palindrome in a text string.

module Main where
import Char

text = "I'll just type in some example text here and embed a little \
\palindrome - A man, a plan, a canal, Panama! - I expect that will be \
\the longest palindrome found in this text.\
\Lorem ipsum dolor sit amet, consectetur adipiscing elit.\
\Integer volutpat lorem imperdiet ante bibendum ullamcorper. Mauris \
\tempor hendrerit justo at elementum. Vivamus elit magna, accumsan id \
\condimentum a, luctus a ipsum. Donec fermentum, lectus at posuere \
\ullamcorper, mauris lectus tincidunt nulla, ut placerat justo odio sed\
\ odio. Nulla blandit lorem sit amet odio varius nec vestibulum ante \
\ornare. Aliquam feugiat, velit a rhoncus rutrum, turpis metus pretium \
\dolor, et mattis leo turpis non est. Sed aliquet, sapien quis \
\consequat condimentum, sem magna ornare ligula, id blandit odio nisl \
\vitae erat. Nam vulputate tincidunt quam, non lacinia risus tincidunt \
\lacinia. Aenean fermentum tristique porttitor. Nam id dolor a eros \
\accumsan imperdiet. Aliquam quis nibh et dui ultricies cursus. Nunc \
\et ante non sapien vehicula rutrum. Duis posuere dictum blandit. Nunc \
\vitae tempus purus."

clean = map toLower . filter isAlpha

palindrome str = str == reverse str

substrings []     = []
substrings (x:xs) = substrings' (x:xs) ++ substrings xs where
  substrings' []     = []
  substrings' (y:ys) = [y] : [ (y:s) | s <- substrings' ys ]

longest []     = []
longest (x:xs) = if length x > length max then x else max
  where max = longest xs

longest_palindrome xs =
  longest (filter palindrome (substrings (clean text)))

main = print (longest_palindrome text)

As a comparison, I translated the program into Ruby. I program predominantly in Ruby these days, and I like it, but the Ruby version is 25 times slower (98 sec. vs. 4 sec.), and it’s 2.4 times more lines of code (31 vs. 13 – excluding the text).

A gain in runtime efficiency, expressive power and multi-core capability is very attractive!

I’m using Ruby 1.9.2 and GHC 6.12.3 on Mac OS X 10.5.8 on a 2.4 GHz Core 2 Duo w/ 4 GB RAM.

ruby 1.9.2p0 (2010-08-18 revision 29036) [i386-darwin9.8.0]
Glasgow Haskell Compiler, Version 6.12.3, for Haskell 98, stage 2 booted by GHC version 6.12.2

TEXT = <<END
I'll just type in some example text here and embed a little
palindrome - A man, a plan, a canal, Panama! - I expect that will be
the longest palindrome found in this text.
Lorem ipsum dolor sit amet, consectetur adipiscing elit.
Integer volutpat lorem imperdiet ante bibendum ullamcorper. Mauris
tempor hendrerit justo at elementum. Vivamus elit magna, accumsan id
condimentum a, luctus a ipsum. Donec fermentum, lectus at posuere
ullamcorper, mauris lectus tincidunt nulla, ut placerat justo odio sed
 odio. Nulla blandit lorem sit amet odio varius nec vestibulum ante
ornare. Aliquam feugiat, velit a rhoncus rutrum, turpis metus pretium
dolor, et mattis leo turpis non est. Sed aliquet, sapien quis
consequat condimentum, sem magna ornare ligula, id blandit odio nisl
vitae erat. Nam vulputate tincidunt quam, non lacinia risus tincidunt
lacinia. Aenean fermentum tristique porttitor. Nam id dolor a eros
accumsan imperdiet. Aliquam quis nibh et dui ultricies cursus. Nunc
et ante non sapien vehicula rutrum. Duis posuere dictum blandit. Nunc
vitae tempus purus.
END

def clean str
  str.gsub(/[^A-Za-z]/,'').downcase
end

def palindrome? str
  str == str.reverse
end

def subs str
  return [] if str.empty?
  y = str[0,1]
  subs(str[1..-1]).inject([y]) do |result, s|
    result << y + s
    result
  end
end

def substrings str
  return [] if str.empty?
  subs(str) + substrings(str[1..-1])
end

def longest strs
  strs.inject("") do |max, str|
    max = str if str.length > max.length
    max
  end
end

def longest_palindrome str
  longest(substrings(clean(str)).inject([]) {|result, str|
            result << str if palindrome?(str)
            result
          })
end

puts longest_palindrome(TEXT)

Update 3/13/11 19:30
Rick’s comment (see his blog post linked to in his comment) regarding the importance of algorithm choice is certainly a valid one – a better algorithm in a slower language may win over an inferior algorithm in a faster language (for large enough datasets). However, what I’m becoming interested in is the fact that one may gain both productivity/power and runtime speed by making wise programming language choices.

In the case of an interpreted language such as Ruby, it’s helpful to “stay in C code” as much as possible. In other words, to favor built-in library routines that have been implemented in C over hand written Ruby code. As much as I like Ruby, this is one of the things that bothers me – the fact that the Ruby code written by the programmer is vastly inferior in performance to the built-in library routines.

I wasn’t planning on refining the Haskell version this soon, but after seeing Rick’s blog post response, I couldn’t resist :)

I should first provide some background info for context. When I wrote the original Haskell version above, I was in the middle of an online programming challenge, and my goal was simply to compute the answer in a reasonable amount of time to move on to the next challenge, so the brief, easily understandable, Haskell version worked great. Hence, my disclaimers in the post regarding the naivity and inefficiency of the solution. The Ruby code in this post was an afterthought simply to see a comparison between identical algorithms. Given that apple & apple comparison, I’m impressed with the brevity and speed of the Haskell version.

Since I’m still very much a Haskell newbie, and short on time, I found an incredible solution by Johan Jeuring. It’s a little bit longer than the Ruby version, but much, much faster. It’s so fast, I had to increase the input quite a bit to get a reasonable comparison – I replicated the original text 23 times and reversed one of the replications to make a fairly long palindrome.

Rick’s Ruby version took 169.4 seconds, Johan’s Haskell version took 0.032 seconds. In other words, Ruby takes over 5,000 times as long to compute the result. Clearly this is an apples and oranges comparison, but I fully expect that a Ruby version using an identical algorithm will take 100 times as long (or longer) to run and would be less concise. Giving up runtime performance to gain programmer power is one thing, but giving up runtime performance and power is a tough pill to swallow.

Here is a slightly modified version of Johan Jeuring’s code. He was also kind enough to provide his code here:

-- Reorganized from Johan Jeuring's solution:
module Main where
import Data.List (maximumBy,intersperse)
import Data.Char
import Data.Array 

text = "I'll just type in some example text here and embed a little \
\palindrome - A man, a plan, a canal, Panama! - I expect that will be \
\the longest palindrome found in this text.\
\Lorem ipsum dolor sit amet, consectetur adipiscing elit.\
\Integer volutpat lorem imperdiet ante bibendum ullamcorper. Mauris \
\tempor hendrerit justo at elementum. Vivamus elit magna, accumsan id \
\condimentum a, luctus a ipsum. Donec fermentum, lectus at posuere \
\ullamcorper, mauris lectus tincidunt nulla, ut placerat justo odio sed\
\ odio. Nulla blandit lorem sit amet odio varius nec vestibulum ante \
\ornare. Aliquam feugiat, velit a rhoncus rutrum, turpis metus pretium \
\dolor, et mattis leo turpis non est. Sed aliquet, sapien quis \
\consequat condimentum, sem magna ornare ligula, id blandit odio nisl \
\vitae erat. Nam vulputate tincidunt quam, non lacinia risus tincidunt \
\lacinia. Aenean fermentum tristique porttitor. Nam id dolor a eros \
\accumsan imperdiet. Aliquam quis nibh et dui ultricies cursus. Nunc \
\et ante non sapien vehicula rutrum. Duis posuere dictum blandit. Nunc \
\vitae tempus purus."

clean = map toLower . filter isAlpha

longestPalindrome input =
  let inputArray      =  listArrayl0 input
      (maxLength,pos) =  maximumBy
                            (\(l,_) (l',_) -> compare l l')
                            (zip (palindromesAroundCentres inputArray) [0..])
  in showPalindrome inputArray (maxLength,pos)

longestPalindromes m input =
  let inputArray =  listArrayl0 input
  in concat $ intersperse "\n"
            $ map (showPalindrome inputArray)
            $ filter ((m<=) . fst)
            $ zip (palindromesAroundCentres inputArray) [0..]

lengthLongestPalindrome :: String -> String
lengthLongestPalindrome = show . maximum . palindromesAroundCentres . listArrayl0

lengthLongestPalindromes :: String -> String
lengthLongestPalindromes = show . palindromesAroundCentres . listArrayl0

palindromesAroundCentres a =
  let (afirst,_) = bounds a
  in reverse $ extendTail a afirst 0 []

extendTail a n currentTail centres
  | n > alast = finalCentres currentTail centres
                   (currentTail:centres)
  | n-currentTail == afirst =
      extendCentres a n (currentTail:centres)
                    centres currentTail
  | a!n == a!(n-currentTail-1) =
      extendTail a (n+1) (currentTail+2) centres
  | otherwise =
      extendCentres a n (currentTail:centres)
                    centres currentTail
  where  (afirst,alast)  =  bounds a

extendCentres a n centres tcentres centreDistance
  | centreDistance == 0 =
      extendTail a (n+1) 1 centres
  | centreDistance-1 == head tcentres  =
      extendTail a n (head tcentres) centres
  | otherwise =
      extendCentres a n (min (head tcentres)
                    (centreDistance-1):centres)
                    (tail tcentres) (centreDistance-1)

finalCentres 0     _        centres  =  centres
finalCentres (n+1) tcentres centres  =
  finalCentres n
               (tail tcentres)
               (min (head tcentres) n:centres)
finalCentres _     _        _        =  error "finalCentres: input < 0"               

showPalindrome a (len,pos) =
  let startpos = pos `div` 2 - len `div` 2
      endpos   = if odd len
                 then pos `div` 2 + len `div` 2
                 else pos `div` 2 + len `div` 2 - 1
  in show [a!n|n <- [startpos .. endpos]]

listArrayl0 string  = listArray (0,length string - 1) string

sampleText s = concat (replicate 8 s ++ [ "x" ] ++ [ reverse s ] ++ replicate 14 s)

main = print (longestPalindrome (clean (sampleText text)))

Here's the relevant change to Rick's Ruby version:


def sample_text str
  str * 8 + 'x' + str.reverse + str * 14
end

puts clean(sample_text(TEXT)).longest_palindrome

Tags: ,

Programming Language Popularity – Part Three

October 8, 2010 in programming | 1 comment

I compiled some programming language popularity statistics in April 2009 and October 2009 . Here’s an update for October 2010:

I made a number of Google searches of the forms below and averaged the results:

"implemented in <language>"
  "written in <language>"

Naturally this is of very limited utility, and the numbers are only useful when comparing relatively within one column since the number of results Google returns can vary greatly over time.

Language Apr 2009 Oct 2009 Oct 2010 Position Delta
PHP 680,000 5,083,500 14,096,000 +3
C 1,905,500 16,975,000 9,675,000 -1
C++ 699,000 6,270,000 6,510,000 -1
C# 349,700 2,125,000 5,132,000 +4
Python 396,000 3,407,000 5,114,500 +1
Perl 365,500 3,132,500 4,675,000 +1
JavaScript 102,700 1,163,000 2,120,000 +4
Java 850,000 5,118,000 1,495,500 -5
Ruby 99,650 227,000 1,426,000 +13
FORTRAN 1,621,000 770,850 0
Lisp Family1 176,507 3,489,650 399,685 -6
Tcl 44,800 382,000 313,400 +5
Erlang 22,285 161,700 188,800 +12
Lisp 61,900 486,500 174,050 +1
COBOL 247,300 166,435 +6
Haskell 22,550 280,500 157,150 +4
ML Family2 29,062 1,003,800 149,005 -5
Lua 13,065 131,800 128,150 +9
Common Lisp 20,600 554,500 112,750 -5
Prolog 17,750 390,500 100,000 -4
OCaml 22,000 343,500 99,050 -3
Scheme 86,450 2,100,000 82,650 -13
Scala 3,570 66,250 65,950 +6
Smalltalk 9,105 187,500 56,950 0
(S)ML3 5,173 590,700 42,130 -12
Forth 6,465 146,450 25,880 0
Clojure 782 62,200 23,525 +3
Caml 1,889 69,600 7,825 0
Arc 6,775 286,500 6,710 -10
Io 1,760 198,500 3,025 -7

1 combines Lisp, Scheme, Common Lisp, Arc & Clojure
2 combines OCaml, (S)ML, Caml
3 summed separate searches for sml and ml

Tags: , , , , , , , , , , , ,

Programming Language Popularity – Part Two

October 24, 2009 in programming | Add a comment

I compiled some programming language popularity statistics in April and mentioned I’d update the results in 6 months, so here they are:

I made a number of Google searches of the forms below and averaged the results:

"implemented in <language>"
"written in <language>"

Language # Results
Apr 09		 # Results
Oct 09		 Position
Delta
C 1,905,500 16,975,000 0
C++ 699,000 6,270,000 +1
Java 850,000 5,118,000 -1
PHP 680,000 5,083,500 0
Lisp Family1 176,507 3,489,650 +3
Python 396,000 3,407,000 -1
Perl 365,500 3,132,500 -1
C# 349,700 2,125,000 -1
Scheme 86,450 2,100,000 +2
FORTRAN 1,621,000 N/A
JavaScript 102,700 1,163,000 -1
ML Family2 29,062 1,003,800 +3
(S)ML3 5,173 590,700 +12
Common Lisp 20,600 554,500 +5
Lisp 61,900 486,500 -2
Prolog 17,750 390,500 +4
Tcl 44,800 382,000 -3
OCaml 22,000 343,500 0
Arc 6,775 286,500 +4
Haskell 22,550 280,500 -4
COBOL 247,300 N/A
Ruby 99,650 227,000 -10
Io 1,760 198,500 +6
Smalltalk 9,105 187,500 -1
Erlang 22,285 161,700 -7
Forth 6,465 146,450 -1
Lua 13,065 131,800 -5
Caml 1,889 69,600 0
Scala 3,570 66,250 -2
Clojure 782 62,200 0

1 combines Lisp, Scheme, Common Lisp, Arc & Clojure
2 combines OCaml, (S)ML, Caml
3 summed separate searches for sml and ml

Tags: , , , , , , , , , , , , , , , , , , , ,

Programming Language Popularity

April 21, 2009 in programming | 10 comments

Despite the numerous ways in existence to quantify programming language popularity, I thought I’d throw yet another one into the mix. I made a number of Google searches of the forms below and averaged the results:

"implemented in <language>"
"written in <language>"

I’m very curious to see how these stats change over time, so I’ve added a calendar item to recompute them in six months. Leave a comment if you’d like to add a programming language to the list, and I’ll update this article and it will be included in the recomputation six months from now.

Language # Results
C 1,905,500
Java 850,000
C++ 699,000
PHP 680,000
Python 396,000
Perl 365,500
C# 349,700
Lisp Family1 176,507
JavaScript 102,700
Ruby 99,650
Scheme 86,450
Lisp 61,900
Tcl 44,800
ML Family2 29,062
Haskell 22,550
Erlang 22,285
OCaml 22,000
Common Lisp 20,600
Prolog 17,750
Lua 13,065
Smalltalk 9,105
Arc 6,775
Forth 6,465
(S)ML3 5,173
Scala 3,570
Caml 1,889
Io 1,760
Clojure 782

1 combines Lisp, Scheme, Common Lisp, Arc & Clojure
2 combines OCaml, (S)ML, Caml
3 summed separate searches for sml and ml
Update 4/23/09 added C#, Tcl per comment requests.

Tags: , , , , , , , , , , , , , , , , , , , ,

Use Ruby to parse NMEA sentences from your GPS

April 8, 2009 in programming, technology | 1 comment

I recently obtained a mobile broadband device that has a built in GPS receiver and can emit NMEA sentences. My old Garmin portable GPS can emit NMEA also, but it’s a pain to hookup to the laptop. Combining a GPS unit in a mobile broadband device is a great idea.

Update: it appears that the accuracy radius of the wireless card is quite a bit larger than my old Garmin unit. The Garmin is usually between 15 and 30 feet, but the Sierra Wireless 598U ranges from 100 to 1,000 feet or more.

After installing the ruby-serialport gem, I was able to write a simple Ruby program to read GPS information from the device and update a remote file on my web server to allow real time location tracking.

Add a simple server side script to read the file and update an iframed Google Map and you’re all set.

The code is also in the Ruby section of my sample code repository on Github.

sudo gem install ruby-serialport

#!/usr/local/bin/ruby
# Author: Brian Adkins
# Date:   2009/04/08
# Copyright 2009 Brian Adkins - All Rights Reserved
#
# Ruby program to retrieve and parse GPS information (via NMEA sentences)
# from a Sprint Sierra Wireless 598U device.
#
# ruby gps-nmea.rb                # prints latititude/longitude info
# ruby gps-nmea.rb update-remote  # scp a file of location info to a remote server
#
# This program depends on the ruby-serialport gem:
# sudo gem install ruby-serialport
#
# From: http://www.gpsinformation.org/dale/nmea.htm#GGA
#  $GPGGA,123519,4807.038,N,01131.000,E,1,08,0.9,545.4,M,46.9,M,,*47
# Where:
#      GGA          Global Positioning System Fix Data
#      123519       Fix taken at 12:35:19 UTC
#      4807.038,N   Latitude 48 deg 07.038' N
#      01131.000,E  Longitude 11 deg 31.000' E
#      1            Fix quality: 0 = invalid
#                                1 = GPS fix (SPS)
#                                2 = DGPS fix
#                                3 = PPS fix
#              4 = Real Time Kinematic
#              5 = Float RTK
#                                6 = estimated (dead reckoning) (2.3 feature)
#              7 = Manual input mode
#              8 = Simulation mode
#      08           Number of satellites being tracked
#      0.9          Horizontal dilution of position
#      545.4,M      Altitude, Meters, above mean sea level
#      46.9,M       Height of geoid (mean sea level) above WGS84
#                       ellipsoid
#      (empty field) time in seconds since last DGPS update
#      (empty field) DGPS station ID number
#      *47          the checksum data, always begins with *

require 'rubygems'
require 'serialport'

# Emacs macro to reset user modified values (highlight, then: M-x eval-region )
# ((lambda (&optional arg) "Keyboard macro." (interactive "p") (kmacro-exec-ring-item (quote ("USERNAME
\"
\372\"HOSTNAME
\"
\372\"REMOTE_DIR
\"
\372\"" 0 "%d")) arg)))

# --- MODIFY THESE -- #
USERNAME   = ""  # Username for remote host
HOSTNAME   = ""  # Remote host name e.g. foo.com
REMOTE_DIR = ""  # Remote directory e.g. /var/www/bar
# --- MODIFY THESE -- #

port_str  = '/dev/cu.sierra05'
baud_rate = 9600
data_bits = 8
stop_bits = 1
parity    = SerialPort::NONE

sp = SerialPort.new(port_str, baud_rate, data_bits, stop_bits, parity)

# lat is of the form 4807.038 where the first 2 digits are degrees and
#   the remainder is minutes.
# dir is either 'N' or 'S'
def convert_lat lat, dir
  degrees = lat[0,2].to_f + (lat[2,lat.length-2].to_f / 60.0)
  dir == 'N' ? degrees : -degrees
end

# lon is of the form 01131.000 where the first 3 digits are degrees and
#   the remainder is minutes.
# dir is either 'E', or 'W'
def convert_lon lon, dir
  degrees = lon[0,3].to_f + (lon[3,lon.length-2].to_f / 60.0)
  dir == 'E' ? degrees : -degrees
end

TEMP_PATH = '/tmp'
TEMP_FILE = 'location.txt'

def update_remote_info lat, lon
  File.open("#{TEMP_PATH}/#{TEMP_FILE}", 'w') do |tf|
    tf.puts Time.now.to_s
    tf.puts "#{lat},#{lon}"
  end
  puts 'Updating remote location info'
  `scp #{TEMP_PATH}/#{TEMP_FILE} #{USERNAME}@#{HOSTNAME}:#{REMOTE_DIR}/#{TEMP_FILE}`
  File.delete("#{TEMP_PATH}/#{TEMP_FILE}")
end

# 99 requests should be sufficient to find a $GPGGA sentence
99.times do
  if (str = sp.gets) =~ /^\$GPGGA/
    fix = str.split(',')
    if fix[6] == '1'
      lat = convert_lat(fix[2], fix[3])
      lon = convert_lon(fix[4], fix[5])
      if ARGV[0] == 'update-remote'
        update_remote_info(lat,lon)
      elsif
        puts "#{lat}, #{lon}"
      end
      exit 0
    end
  end
end

puts "Invalid data - GPS coordinates not found"

Tags: , , ,

Retrieve Sunrise, Sunset & Twilight Info in Ruby

March 11, 2009 in internet, programming, science, technology | 4 comments

Sunrise, Sunset & Twilight

TwilightI was curious about the exact time of sunrise & sunset at my location, so I found this US Naval Observatory site. In the process, I learned a more precise definition of twilight. I wanted to be able to automate the process of retrieving the information, so my first attempt was to simply put the query parameters used in the form in the URL as an HTTP GET request, but the server wouldn’t accept that, so I needed to issue an HTTP POST request.

Ruby Code

Ruby is a great language for this sort of task, so I put together the following simple program:

require 'net/http'

YOUR_ID    = ''    # A unique ID per comment above
YOUR_CITY  = ''    # The name of your city
YOUR_STATE = ''    # Two letter state abbreviation

now   = Time.now
month = now.month
day   = now.day + 1 # Tomorrow
year  = now.year

Net::HTTP.start('aa.usno.navy.mil') do |query|
  response = query.post('/cgi-bin/aa_pap.pl',
    "FFX=1&amp;amp;amp;ID=#{YOUR_ID}&amp;amp;amp;xxy=#{year}&amp;amp;amp;xxm=#{month}&amp;amp;amp;xxd=#{day}&amp;amp;amp;st=#{YOUR_STATE}&amp;amp;amp;place=#{YOUR_CITY}&amp;amp;amp;ZZZ=END")
  if response.body =~ /Begin civil twilight[^0-9]*(\d+:\d{2} [ap].m.).*Sunrise[^0-9]*(\d+:\d{2} [ap].m.).*Sunset[^0-9]*(\d+:\d{2} [ap].m.).*End civil twilight[^0-9]*(\d+:\d{2} [ap].m.)/m
    puts "#{month}/#{day}/#{year}"
    puts "Begin Twilight: #{$1}"
    puts "Sunrise       : #{$2}"
    puts "Sunset        : #{$3}"
    puts "End Twilight  : #{$4}"
  end
end

You just need to edit the three constants that begin with YOUR_. The id used on the Navy web form is ‘AA’, but they have a comment in the HTML that requests you use a unique id of your own up to 8 characters to help them with tracking. You can find a more complete version of the code in my github profile.

Emacs Goodness

After writing the above Ruby script, I made it executable, ‘chmod +x sunrise.rb’, and placed it in my path so I could write a simple Emacs function to invoke it.

(defun bja-sunrise ()
  "Display sunrise, sunset &amp; twilight information."
  (interactive)
  (shell-command "sunrise.rb"))

Imagine my surprise when I invoked the Emacs apropos help ‘C-h a’ to see my newly defined function and discovered that Emacs, naturally, already has several commands to display sunrise/sunset information!

calendar-mouse-sunrise/sunset
Show sunrise/sunset times for mouse-selected date.
calendar-sunrise-sunset
Local time of sunrise and sunset for date under cursor.
sunrise-sunset
Local time of sunrise and sunset for today. Accurate to a few seconds.

It doesn’t, however, display twilight information, so my simple function still has a purpose in life. Emacs is awesome :)

Tags: , , , , , ,

« Older entries