Modeling with Data: the blog

Testing the model using the model

Ben Klemens — 11 January 2010 00:00:00 +0000

I start by picking at a cartoonist, but this goes well beyond that in importance.

Keeping paper current

Ben Klemens — 9 January 2010 00:00:00 +0000

In which I put an absolutely huge amount of thought into things most folks don't really care about. Then, I exert great effort fighting against the inevitable.

The statistics style report

Ben Klemens — 19 December 2009 00:00:00 +0000

I like to think that I always lean toward parsimony, but I imagine others looking at my work wouldn't think so.

Git status interactive

Ben Klemens — 12 December 2009 00:00:00 +0000

This is a useful script for people using git; if that's not you, don't bother clicking through. I know I've been writing a lot about computing technique lately, so next time: statistics!

The schism, or why C and C++ are different

Ben Klemens — 21 November 2009 00:00:00 +0000

Take this as a solicitation for suggestions on how we can get Apophenia's variable-length lists of named arguments (which are entirely C standards-compliant) to work from C++.

Yet another Git tutorial

Ben Klemens — 8 November 2009 00:00:00 +0000

I wrote this for my coworkers, upon whom I've just dropped this whole revision control thing, because the existing tutorials didn't seem at all appropriate for them.

Easy re-typing with designated initializers

Ben Klemens — 1 November 2009 00:00:00 +0000

A straight-up lesson on coding technique. Some of this should have been in the texbook, but I was somehow reluctant to rely too much on designated initializers and compound literals; in retrospect, I'm not sure why.

Computing history and its scars

Ben Klemens — 19 October 2009 00:00:00 +0000

Although it may seem like I'm complaining about whatever glitches, in an alternate universe we'd just have different scars. The path-dependency and potentially infinite life of code means that software is inherently prone to scarring.

Causal ethics

Ben Klemens — 11 October 2009 00:00:00 +0000

Yes, this is a blog about statistics, and yes, this is a post about the development of ethics.

Programming your blog

Ben Klemens — 21 July 2009 00:00:00 +0000

When I'm having trouble with a machine of any sort, I try to think of it as a cultural problem. How were the designers thinking differently from how I'm thinking?

Probability versus likelihood

Ben Klemens — 2 July 2009 00:00:00 +0000

The question of whether data is plural or a singular can be reasoned through; the question here is well on its way to unanswerable.

Data is typically not a plural

Ben Klemens — 10 June 2009 00:00:00 +0000

This is a lead-in to next episode about another vocabulary problem that's much more difficult.

Better variadic functions in C

Ben Klemens — 3 June 2009 00:00:00 +0000

A new edition of the C standard will only come in the distant future (perhaps never). In the mean time, here's my best solution to what I see as the language's most glaring problem. I'm open to suggestions on streamlining it further.

Alternatives to Word

Ben Klemens — 30 May 2009 00:00:00 +0000

In two episodes, I'll tell you what I use for this here web site.

Word and standards

Ben Klemens — 22 May 2009 00:00:00 +0000

I once toyed with writing a book about computing standards, as a follow-up to my exciting book on software patents.

The ergonomics of the down arrow

Ben Klemens — 19 May 2009 00:00:00 +0000

This one may seem odd, because it's not a subject many people (including designers) spend much time talking about, but it can still make a big difference, and points to the value of at least having a choice.

Intuition versus ease of use

Ben Klemens — 15 May 2009 00:00:00 +0000

Ease of initial use is a distinct concept from ease of long-term use. Sometimes they are in happy agreement, and sometimes the first blocks the second.

Views on a paradigm

Ben Klemens — 11 May 2009 00:00:00 +0000

Part two of six on efficient document prep techniques that are precluded by Word's design. Don't forget: if you don't have time in front of the computer to read this web page, you can print the PDF.

Why Word is a terrible program

Ben Klemens — 7 May 2009 00:00:00 +0000

This is an essay from late 2005, in six parts. I'm reprinting it here because I think it's a good vehicle to get across a number of points about design of information and software.

A general model object

Ben Klemens — 4 May 2009 00:00:00 +0000

Last time, I talked about the important problem of balkanization in modeling; this episode is about doing something about it in code.

Bringing theoretical models to data

Ben Klemens — 1 May 2009 00:00:00 +0000

It's amazing how very differently people can read a phrase as simple as "Modeling with data".

Operator overloading

Ben Klemens — 29 April 2009 00:00:00 +0000

Sometimes human language and mathematical language fall into conflict.

Making integers two-dimensional

Ben Klemens — 22 April 2009 00:00:00 +0000

Or, a means of visualizing integer arithmetic.

Using a program as a library

Ben Klemens — 13 April 2009 00:00:00 +0000

I'd been neglecting computing in favor of statistics, so here's a useful tip for managing your code.

Your genetic data---ethics

Ben Klemens — 8 April 2009 00:00:00 +0000

I'll get back to this in a few episodes, but for now, the point here that the imbalance between descriptive and inferential power has ethical as well as purely research implications.

Your genetic data

Ben Klemens — 6 April 2009 00:00:00 +0000

Given a hundred cases and controls and their genetic markers, I can find you markers to perfectly categorize them in under two minutes. Of course, those markers provide no predictive power outside those hundred subjects.

Freakophenia

Ben Klemens — 3 April 2009 00:00:00 +0000

The summary of today's entry: if the research is exciting enough to be written up in the newspaper, it's probably not to be trusted.

Breaking the pipeline

Ben Klemens — 30 March 2009 00:00:00 +0000

Much of this one is about the requirements for the OLS coefficients to provide the line of best fit (i.e., the Gauss-Markov theorem). If you want to play along, try to recall them before you click through. E.g., must the expected value of the errors be zero? Are errors assumed to be Normally distributed?

Crowdsourcing data mining

Ben Klemens — 24 March 2009 00:00:00 +0000

I think that this gets to the question of what exactly is a hypothesis test. That's a question that we don't seriously have an answer to, though I may offer some comments at a later date.

Dataviz

Ben Klemens — 18 March 2009 00:00:00 +0000

This is cut from a presentation I gave last November (so if you have time to kill, have the right plug-ins, and want to hear my voice, here's a link).

Too many tests

Ben Klemens — 16 March 2009 00:00:00 +0000

Today for lunch, I had rice with wheat gluten, tomatoes, and cashews. The rice is somewhat sweet and very purple, but I can't tell you what kind it is exactly, because the package is entirely in Korean.

Parallels II

Ben Klemens — 13 March 2009 00:00:00 +0000

This is a continuation of last episode, discussing how technology affects the math we do. Next time, I'll switch from computing to conflicts in stats.

Moore's law won't save you

Ben Klemens — 8 March 2009 00:00:00 +0000

A lot of us are stuck in an optimistic early-90s mentality toward progress in computing power.

Today I am a blog

Ben Klemens — 6 March 2009 00:00:00 +0000

I never really know what to say in these introductions.