Wind Data gripes

Well first let me start off by saying, any and all wind data is helpful….

I’ve been working with HRD going on 12 years now, and for most of that time (like the last 11) one of my main tasks has been data collections. This include collecting information about particular stations as well as the data itself. Every year before the start of hurricane season (June 1st for the Atlantic) I go through all my reference tables and check and add any new stations that have come online since the previous year (usually I do this a couple times a year, but always right before the season starts). So this year like all the past years, there are a number of new stations. First yay!, second…. ugh… Why I say ‘ugh’ well 1 it means that I now have to research new stations and collect information for them, and 2 it reminds me of how many stations, well how do I put it… are installed by … (this is a family safe blog), but I have to hope that the installer has no formal meteorological training.

But again I must say, all data is greatly appreciated. But when you install an anemometer on the roof of a building or house or near a tree, wall or anything that can change the flow of air to the anemometer… well lets just say that you basically make your data almost useless. When anemometers are installed they should be in clean air, that means a safe distance from any upwind obstacles. And no the roof is not a good place, unless you are going to install a tower tall enough to avoid the affects the roof actually cause, especially roof with a good steep angle. Depending on the location, winds can have increased values or in many cases where they are partially blocked, very low poor values. Also if you are wonder what a good height is… where the National Hurricane Center defines a surface wind at 10 meters, so if you can get you anemometer up 10 meters even better.

Now before all you amateur meteorologist start sending me hate mail… well let me state that the only reason I’ve writing this post is because these sites that I’m complaining about are installed by public sector groups, not private citizens. In most cases, I have no access to private data collected by home owners and amateur meteorologist, I only have access to data on the international feeds. So now do you see why I’m so frustrated with the installations….

All I ask is that you think before you install. And yes there are standards… take a look at http://www.ofcm.gov/siting/text/a-cover.htm if you are serious about this kind of thing.

But again, for all you out there providing wind data. THANKS! and yes I do mean it… any data is better than no data.

Python make pretty pictures

Well as those who know me, know that I’ve been on sort of a Python kick lately. A few months ago I started learning Python. So now that a few months have past I’ll share the latest reason I like Python… Graphics….

Early in my time with HRD I started working with graphical products. Initially I was playing with a product called IDL. I thought the product was great, but we had a mathematician in join our group who was way more advanced with IDL than I and I let him take control of the product and was very happy to move on to other projects. This said I’ve always been interested in computer graphics as far back as I can remember… in grade school it started with Logo, and then Graphics on an Apple IIe…. Back to the some what present… even in grad school I took a course on OpenGL, even made a nifty 3d maze game. Anyways again I digress (pretty typical of all my blog posts…)

So graphics has always been something I’ve wanted to get into, plus with the always evident idea in my group of making our product portable and deployable, I also new one day that I’d have to find a non-IDL/commercial solution. I state non-commercial as well since I could also have looked at MatLab or such. Anyways the first two products that leapt out at me a while back were GDL a GNU clone of IDL and Octave, another GNU project compatible with MatLab. The only problem was that GDL at the time did not support many of the IDL tools we were now using, and Octave… well I still don’t know MatLab so that wasn’t going to work.

Anyways…. so I place the idea of a free and distributable version of our product tools on the very back burner… you know the one that was never even attached to the stove…

So we move foward in time to the real present… Like I said in the beginning of the post, a few months ago I started learning Python. Like learning any new programming language, it is mostly syntax and package/module discovery. The syntax part was straight forwards, I’m a C/Fortran/Java/Unix scripter…. so yes, Python syntax, not anything new. As for packages… well they are abuyndant, but easily managable. Unlike Perl which I don’t even know where to start on how large it has gotten (and yes in my early days I was even a Perl programmer), trying to find the ‘best’ package can take as long as trying to write your own… So as for Python, I quickly learned to read and write ASCII files and perform some simple mathematical calculations on data and a few other simpler tasks.

Then a co-worker of mine asked me about writing a program for a project he was looking into. The project seemed straight forward enough… read in a couple years worth of data (all of which were simple 2d matrixs), perform some simple stats on them (mean, std, median, min, max), spit out a simple ASCII summary, and generate some pretty pictures. All of the first steps seemed straight forward, the pretty pictures… well that was a different story. Anyways still being on my Pytohn kick, I quickly started out on this new project in Python. Took a couple of days to get the files read and stats processed, was even easier when I learned about NumPy and SciPy. NumPy is great, it provides great array handling for large matrixes and such. Plus now the reason for this post… an additional package called MatPlotLib is also part of the NumPy/SciPy family. MatPlotLib made plotting the matrixes an almost breeze. Basically once I figured out my dimensions and the geospatial area for my map to plot on, all I had to do was give it my NumPy array and say draw, and poof…. pretty picture.

Don’t ask me what the picture is exactly of, but ain’t it pretty? I do know that it is some sort of thermal data from the GOES satellite (or atleast I think it is from the GOES satellite). But ain’t that cool… all I did was create Basemap and then tell it to plot using the pcolor(…) command. It was that simple. Another reason I think Python and I are going to have a long happy relationship. A few more tweaks and I should have my program fully ready to go… only issue is… currently it is kind of slow.

XML Modeling

Well I’ve been trying to teach myself XML over the last few weeks. Overall I don’t see it as that difficult. I’ve worked with XML data in the past but usually through various interpreters and other applications. This has been my first ‘experiment’ with developing and XML Schema.

Modeling XML is a pretty straight forward idea, and pretty simple… now doing it right…. So anyways I initially started out by buying a book, after reviewing it from the library I picked up XML By Example by Benoit Marchal for <$5 on Amazon. Then without touching the computer I first read the first 4 chapters and briefly skimmed the rest of the book. I stopped after Chapter 4 since it is on XML Models. So at that point I began my first model… well I think I bit off more than I could chew. I went for designing a model to fit the entire set of formats found in the H*Wind project. Well like I said, much more then I could chew….

So I went back and decided just to model a single ‘type’ I chose the Storm Center/Forecast type since it also coincides with another project we are starting. Well I first went through the data file and decided which fields we needed for our purposes. I then defined my ‘variables’ or elements. I then started putting things together. Next I tried to build a sample data file based on the schema… what I found… my model sucked. So back to the modeling stage…. well I can see why this is not so simple now…

Well so I think I’ll give it one more modeling try and then I might start looking for an app to reverse model an existing XML file…. I think my main issue may just be in the abstractness of modeling without data.

Job Description vs Actual Job

I reluctantly write this post here since I’m not sure who actually reads this blog and how it will be taken. I roughly know who reads the Lauren and Nick Blog, but I’m not totally sure who reads this one. I know it is published via my Plaxo Pluse and I think I have it imported into my Facebook Notes, but I’m not sure nor am I sure if I have it being published anywhere else…

Anyways, this topic came up among friends last night. As the post title states, the wonder is what others think about there Job Description as compared to what they actually ‘do’. More importantly performing the role of a manager when it is not in your job description.

Basically when you think about it we are ‘only’ paid to do what is stated in our job descriptions. So what do you do when you find yourself doing other work. I’m not saying that you find yourself doing the work of others, but that your work has out grown your job description. Say you were hired as a junior level engineer, but over time while your position has not change, you now find yourself managing a small project. Or say you are a entry level financial analyst but find yourself doing the same work as your superiors. Or even say your job description includes the work of a upper level manager but you title only lists you as a junior level manager…. anyways these are all very common possibilities… so what should you do?

do you:

1. Accept it and keep going
2. or Accept it and seek redefinition of your job description
3. or Accept it and simply seek compensation (raise, bonus, etc)
4. or Reject it and simply not do it

All very plausible options but which should you take. Now there are also other factors:

1. Do you want the work?
2. Are you seeking advancement or raises?
3. Do you feel you are not capable of the doing the work (not likely if you are already doing it)

Regardless of the motive, desire of personal choice, I guess the bottom line question is, do you do it or not? This means until our job description matches our job, or do we wait till our job description changes and then start the work? So what does everyone else out there think?

A workspace only a geek would love.

Exluding the really messy desk, I’m loving my workspace. Between VNC, a utility called Synergy (http://synergy2.sf.net), and my new found friend ScreenRecycler (http://www.sceenrecycler.com) for the Mac, I am happy. (For windows, you can try http://www.maxivista.com/ for Windows)

VNC is a great utility for remote displaying computers. I use TightVNC on linux and Vine Viewer on the Mac. Via VNC I am able to display my linux servers in Miami locally on my mac in DC.

Synergy is a great little tool tha let’s you share one keyboard and mouse with several machines. It works for Linux, Mac, and yes even windows though I never tried it. Synergy let’s me use my keyboard and mouse on my Powerbook without any addition hardware, plus it is free.

And today… Today I found a great app for the Mac call ScreenRecycler. SR let’s me use a VNC session as a second monitor. Basically on my linux box using a vnc client I connect to my mac. That connection the becomes a second monitor for my mac. If you look closely at the center screen you will see my second mac window, typically I’d have the screen set to full, but for the picture it is better this way.

OhGizmo! Archive Prison Pen – Useful For Writing, Not For Shivving

We had these at work a few years ago, and while as cool as they were, they wrote like crap and dried out even faster. That and if you left them in the sun, a few days later the plastic/flexy part would dry out and then break. But they are cool looking.

OhGizmo! Archive Prison Pen – Useful For Writing, Not For Shivving

SSH and Permissions

I just spent the better part of the last 2 days working on setting up SSH with private/public shared keys. I’ve done this many a times so I launched my nifty file containing the documentation I’ve used in the past and went to work. I first made my key pairs on both machines, and then I transfered the file to there respective sides. The connection for hostA to hostB worked flawlessly from the first instance, now hostB to hostA was another story. Of course the first thing I did was check the permissions:

hostA:> chmod 700 ~user1/.ssh
hostA:> chmod 600 ~user1/.ssh/*

and tried again… still did not work. Next I checked the /etc/ssh/sshd_config , everything looked fine and actually when diff’d with the one on the other server, they were identical. I should mention that hostB is a Redhat ES 4.4 server and hostA is Centos 4.4 server (basically identical). Well I scratched my head and went to google. Of course every forum and mailing list result I found said: “check the permissions”. Well I did it again….. and again… and again…. still no luck. This is when I decided to try a different user.
hostA:> chmod 700 ~user2/.ssh
hostA:> chmod 600 ~user2/.ssh/*
And guess what…. it worked. this is when it hit me. the actuall directory structure. The typical hierarchy looks like this:
owner path
root /home
user1 /home/user1
user2 /home/user2

but this was my hierarchy:
owner path
root /home
user2 /home/user2
user1 /home/user2/user1

All I can figure is that since the parent to the users home was not owned by root, SSH didn’t work. Solution:

owner path
root /home
user2 /home/user2
user1 /home/user1

ln -s /home/user1 home/user2/user1

Argh…. 2 days wasted and while I swore it was not permissions, it was just under the name “ownership”. All because for layout reasons I want the home directory of one user with in the other.

First Aid and CPR

Today is CPR and first aid trainig at work. I always like practicing on partial bodies. After lunch we get to do the first aid part.

Holiday boredom

Well for those who know me… I’m an associate employee of NOAA at AOML. Of course this
means I’m stuck at work with limited days as actual holidays. But for those who know…. this is the time of year when all the federal employees have start burning their use-or-lose vacation time. Federal employees can only carry over a certain amount of hours from one year to the next, all those extra hours are there for lost, as a result…. from about December 1st to January 8th this place is like a ghost town. At last count there are 7 people in my wing of the building which is composed of HRD. So while everyone is vacationing… I’m sitting here rather bored… don’t get me wrong, I get a lot of work done this time of year… but I like to see people every now and then. oh well back to my work……

Self pride

Wow, I read engadget everyday, who ever think that I’d be working on a project that they post. I’m working on NOAA’s Hurricane Research Division’s (hrd) side bringing the data into a visual app we developed.

link: NOAA to send unmanned aircraft into hurricane – Engadget