Writings

Technology, open source, personal essays, and everything that isn't climate.

NYTimes: Who owns that science data

NYTimes has an decent article on some of the data sharing rules around science in general, and the Kepler mission in specific.  The Kepler team wants an extra 6 month embargo on data sharing because there were some delays in getting rolling.

Compare and contrast the 12 month embargo on data sharing, and the fact that extending it to 18 months is enough to warrent a Times article vs. the 17 year exclusivity of software algorithms.  I wish people would get more riled up about the later.

Related: NYTimes on Bard's Citizen Science · Mashable: Who Owns Your Data? · Science Blog

Teaching Engineering in Kindergarden

Glenn Rock, New Jersey did an inspired thing, and added Engineering into their kindergarden program:

They plan multiday projects, often built around classic and popular stories like the Three Little Pigs, and take students step by step through the engineering process: design, build, test, evaluate. “They have to have the thinking skills of an engineer to keep up with all the innovation that’s constantly coming into their world,” Ms. Morrow said. First graders were recently challenged with helping a farmer keep rabbits out of his garden. In teams of four, they brainstormed about building fences with difficult-to-scale ladders instead of doors and setting out food decoys for the rabbits. They drew up blueprints and then brought them to life with plastic plates, paper cups, straws and foam paper. Then they planned to test their ideas with pop-up plastic rabbits. If the fences were breached, they would be asked to improve the design. “It gets your brain going,” said Elizabeth Crowley, 7, who wants to be an engineer when she grows up. “And I actually learn something when I’m doing a project — like you can work together to do something you couldn’t do before.”

I’m hoping the Race to the Top program spurs more of this kind of thing.

Related: Software Engineering Talk at Vassar College · Tools vs. Process · I never thought that one of the skills I'd need to have as a farmer is PHP

The 11th Doctor

The 11Th Doctor

We're nearing the end of Doctor Who for this year, after the year of Dr Who hiatus. I had become such a David Tennant fan during his reign as the Doctor that I had severe trepidations on what a new Doctor would bring. While I knew we were getting Moffat as a show running, Matt Smith just didn't look the part to me. After 11 episodes, I have to say, I'm enjoying this year of Doctor Who possibly more than any other since the return to TV.

Matt Smith is just awesome. He's a more alien doctor, like Tom Baker was. He's very clearly not human. He's also not quite as invincible as the Doctor has been in recent years. There are things that actually scare him, and that fear is felt by the audience. That real sense of fear means real tension, and the belief that things might not always turn out right in the end.

This season also finally embraces the rich history that is Dr Who. We never got a reference to past incarnations of the doctor until this year's season premier. We are finally getting glimpses of past doctors (I think we've seen William Hartnell's picture 3 times this year), which puts Dr Who back in the context of the much larger history of the series, instead of shooing it off like Russel Davies did. Getting the Silurians back this year was definitely part of that.

I eagerly await the season finale, which Moffat himself will pen.  Given everything great we've gotten so far, I can't wait to see how we're going to wrap this up this year.

Related: Fish Custard and the year of Doctor Who · Doctor Hoo · Doctor Who Floating Tardis

Pair Programming, why it works

I was recently forced into a pair programming exercise at work because we'd gotten to a critical bring up part of the code where nothing was really able to be done in parallel. Given that we are all noobs on this source base, there also wasn't really anyone that should just go take the lead on it either. Given that 1 member of my team is actually in the same office as I am, we decided to try pair programming through it, because what else were we going to do. Jeremy brought in his laptop, and he mostly drove while I navigated as he was writing code. The results were impressive. In 2 days we got from a server that wouldn't start, to one that was responding to half the nfs commands.

When first presented with Pair Programming most developers / managers have the same response: I'm going to have 2 people to do 1 person's job?  That's just crazy inefficient.  I admit, that I had my doubts, but as there is a decade of success behind this technique I decided it was worth trying.  The generic "it's actually more efficient" statement didn't sit well with me because there was not explanation behind it. The why it works is actually really interesting, and I think it also explains the types of problems it is good for and the ones it is not.

Anyone that's developed software has run into programmers rut at some point.  This isn't the same thing as writers block, it's not the inability to create something, it's having something not quite work and looping for hours to try to figure out why.  Typically it's solved by going away from the code for a while (often overnight) and the solution is obvious in the morning.  When tackling a complex problem, you start building a model in your head that matches the software you are writing.  Sometimes you validate that model as you go, but often you don't.  After hours working with that same piece of software you've made hundreds of assumptions on the model, any of which might be wrong.  But once it's fixated in your head, tearing it back down to validate each and every step seems defeatist.  There is "just one more thing" that you could try and it will get you past this block.  How many times have you said that to yourself... just one more thing.  These "just one more thing" loops can easily become 4 hours in a shot.

When you walk away and come back, assuming your brain was focused elsewhere at the time, you've got to rebuild that model.  It's not actually an exact copy of the model before, that's why it takes time to get rolling on a complex software task.  And, as often as not you make slightly different assumptions this time, and low and behold you easily see where you were wrong.  Problem averted, and we move on.

But in pair programming, you don't have 1 model, you have 2. At a superficial level, those are the same model, that's the architecture of your project. But on a deep level, the level of actually assigning variables and doing test conditions in the code, those models are different. Those differences are why we enforce code conventions on software, because left to our own devices those differences in mental models would spill into the editors in somewhat surprising ways, making the software itself completely incomprehensible. Look at someone's collection of private scripts they've written to automate small tasks, stuff they never intended to be shared, and you'll see what I mean.

With only one person at the keyboard, and 2 mental models of the code in effect, you'll get a nice amount back and forth when the 2 models don't line up.  It's kind of like unit testing for your brain.  During this exercise there was a lot of "wait, why are you doing that? ... isn't that supposed to be this other value? ... are you sure that value is actually initialized in the called?"  I have no idea how many times I was right or wrong vs. Jeremy, it doesn't matter.  What mattered is that we were constantly checking each other.  I mentally noted at least a dozen times where left alone, I would have been stuck for hours.  While not perfect (we had one rut that got us stuck for 2 hours because we'd both made the same bad assumption in our model), it seemed like we eliminated 90% of our ruts by having the second mental model.

And that's where the efficiency comes from.  On anything other than rote code, software development is not continuous output.  It's some number of hours hours stuck in a rut, then an hour of speedy implementation, then a new rut, then a burst.... rinse, wash, repeat.  It's a set of fits and starts that as developers we convince ourselves was a much smoother process because we felt busy, or just because it's the only way to do it.  But it turns out those fits are actually huge inefficiencies in the process.  They are probably one of the primary causes behind the observation that "great developers" produce 10 - 100x more output than average developers: great developers figured out how to pop out of ruts faster (either consciously or subconsciously). Pair programming gives you a simple tool to get some of that efficiency with your existing team and developers, making everyone a bit greater in the process.

Related: Programming is Hard · What is computer programming? · Software Engineering Talk at Vassar College

Netflix Culture Presentation

There is a presentation out on slideshare on the Netflix culture, and how they treat their employees.  Definitely interesting reading, to see where there are strengths and weaknesses compared to your own organization.

I think the most interesting part of it is the observations starting on slide 45 about talent vs. complexity.  It meshes quite well with what Google has said for years that people tend to tune out: the #1 problem in an organization to hire the best people.

Related: What was behind the New York Times Netflix infographic · Did Netflix not anticipate this? · Presentation Philosophy

Will you pay for news on the web?

I finally have the answer as to whether I would pay for news on the web, and the answer is yes.  Last night after reading the 5th zero content vapid gadget news story that was front page content on wired.com, I realized how much I really appreciate the quality bar that's been set over at Ars Technica.

While most web outlets seem to be degrading in the content they put out there, Ars seems to just be getting better.  They have some quite in depth writing on most of the science and tech space, and aren't afraid to dive deep into subjects with original research, not just falling back on the lazy opinion model that most others have.  I also realized that while not having wired around would mean nothing to me, loosing Ars would be something I'd actually really miss.

Ars's pay model is simple.  If you by a premier account ($50 / year, so roughly magazine cost), you stop being presented with ads on their site, you get access to stories slightly ahead of the public site, and you get personalized rss feeds which provide full stories (their free rss gives you just the first 2 or 3 paragraphs).  There are some other benefits, but the full rss and just knowing I'm helping to keep Ars around is what I care about.

That second point is key.  If the news industry wants people to actually pay for things, they need to stop racing to the bottom on cost, and start racing to the top on quality.

Related: A Climate Change Primer at ARS Technica · Stop and Pay Attention · Ars: Disruption - how one webcomic welcomes the future that so many fear

Neil DeGrasse Tyson on Science Literacy

This is a point that you've started to see resonating recently among both economics and science folks, Thomas Friedman is famous for it as well.  Science illiteracy is the biggest threat to the future of this nation, because it is the science and technology innovation engine that has made us the economic power that we are.  If we want to stay a great nation, we need to keep that engine running.

The video was recorded at this year's world science festival in NYC.  I'm really going to have to get down there next year.

Related: Colbert and Tyson · Why we need more focus on Science · Science Blog

When is Sunrise anyway?

Actually, the amount of refraction changes with air temperature, pressure, and the elevation of the observer (see Chapter 16).  A change of temperature from winter to summer can shift the times of sunrise and sunset by about 20 seconds in mid-northern and mid-southern latitudes.  Similarly, observing sunrise or sunset over a range of barometric pressures leads to a variation of a dozen seconds in the times.

It turns out that something as "simple" as calculating Sunrise is anything but.  After you've worked out all the geometric positions of the planets, you then have to run your simulation backwards due to the ~8 minute light delay, and then you've got to take the pesky atmosphere into account, as it changes the apparent position of everything over the course of the sky.  The effect is non uniform, which is why the Sun on the horizon looks squashed.

Working out all this math (with the help of my trusty copy of Astronomical Algorithms), and putting it into code for Where is Io, remains a taxing mental exercise, but one I am immensely enjoying.

Related: Sunrise at Kennedy Space Center · Walkway Over the Hudson at Sunset · Coming soon to Where is Io

A smart phone vs. a cloud access point

On some feed I came across: How to get rejected from the App Store, and as I read through it I became more and more glad that I've got an Android phone.  Some of the top things that I do with my phone are explicitly prohibited by Apple.  Streaming internet radio, directly syncing podcasts to the phone, having widgets on your desktop, improving on the phone built ins (in this case calendar display), all of these things are prevented on the iPhone.

Yes, the iPhone has a more consistent UI.  It's easy to be consistent if you limit your functionality, and require that everyone that owns your device runs your client software on a desktop in your house as well.  Android phones don't ever assume that.  An Android phone is more than a smart phone, it's a cloud access point.  If you have to use a cable to put data (contacts, music, whatever) on your device that has always on wireless networking... you have failed.

I'm glad that Apple opened up this market for more vendors to play in, but I'm seriously glad that Google is relentlessly pushing it forward.  The post PC era is really about whether or not you need a PC to use your other devices.  As far as I'm concerned, the answer should be no.

Related: I've got an Android in my pocket · Exciting times in mobile · Things to think about if you want an Android phone