Wednesday, 4 July 2012

"No flashy. This is Science." [censored] -- Or: How two jetlagged IR researchers met and had an idea

[Prologue: the beginning of a beautiful friendship]

J. Hi. Can I sit here?
S. Sure. Hi. I don't think we met before. Nice to meet you, I'm S.
J. I'm J.
S. Nice place here, isn't it?
J. Yep! Beach this afternoon?
S. Sure!
J. Ok, let's pretend we do some work first. So, what are you working on?
S. Oh, several things, blablabla [snip] And you?
J. Well, I did blablabla... and blabla... and I also published some papers on formal analysis of clustering metrics.
S. Interesting. I also started something similar for IR metrics years ago, but I never managed to publish it.
J. Really?
S. Yep. I'll show you [opening his laptop]. See, I defined a framework based on measurement theory, then I defined some axioms...
J. That's crazy! I did exactly the same!
S. ... then some desiderata...
J. Exactly the same!
S. ... Yes but you published, I didn't...
J. ... because you tried to do everything, look here, that's crazy!
S. Yes, I'm a bit, sure... and then blablabla...
J. blablabla [very technical details here - ok, ok, I admit I do not remember that!]
S. blablabla [very technical details here - ok, ok, I admit I do not remember that!]
J. Well, why not proposing this as an idea for SWIRL research directions?
S. We might indeed!
J. Ok, beach time now.
S. Beach!

[Chapter 1: split groups doing Science]

M. So, let's have a round of the table and everyone presents his own idea.
[various good ideas...everyone presents just one, accordingly to the rules. But you know that Italians and Rules do not fit well in the same sentence, so...]
S. I do have two ideas. The first is not very exciting, but it's Science. The second is more exciting...
Others. Well, tell us both. [meaning: the usual Italian breaking rules...]
S. The first is about IR effectiveness metrics. We have around 100 metrics, counting the system-oriented ones only. The research project would be to find formal properties that each metric should satisfy.
Others. So do you mean...?
S. To define axioms that metrics should satisfy. For instance, first retrieved documents should weight more (or not less) than following docs, etc.
Others. Hm. What's that for?
S. Oh well, for instance we could understand why nDCG discount function is defined in that way...
Others. Cool. We need a name...
A. Axiometrics!
Everyone. That's great!!

[Chapter 2: Axiometrics]

S. Hi J.. Did you mention the idea about metrics in your group?
J. No, I didn't, and you?
S. Yes, I did, and guess it, it was selected!
J. Really?? Great!!
S. And A. invented a cool name: Axiometrics
J. That's wonderful!
S. Let's have some coffe, I'm still jetlagged.
J. Yes. And beach later. And beers.

[Chapter 3: Coffee and beers]

[after some hours, and coffee. And beach. And beers.]
S., J., A. M. N. Ok, let's write this short report about Axiometrics in SWIRL...
... well, let's have a short and last beach session first!

[Chapter 4: Planes, emails, and deadlines]

[After some long flights back home, tons of unanswered emails and student requests, expenses claim forms, etc. etc., and just a few spare days before the deadline...]
S. J., do you think it is reasonable to submit a Google research grant proposal on Axiometrics? Or is it a stupid idea?
J. That's a great idea!
S. Let's involve E. as well, he is interested.
[some hard science follows: bibliography is polished, CVs are created, margins and fonts are modified, PDFs generated...]

[Chapter 5: 4th of July]
Guys, we won!!

P.S. Thanks to Julio, Evangelos, ArjenMarteen, Nicola. And to SWIRL and Mark!


No comments: