IRC office hours/Office hours 2010-09-22

Philippe: Just a reminder, this chat will be publicly logged, starting.... now

[3:59pm] basile: hmm, I got disconnected again

[3:59pm] basile is now known as guillom.

[3:59pm] Philippe: And the logs will be posted at meta

[4:00pm] BloodRedSandman joined the chat room.

[4:00pm] jorm joined the chat room.

[4:00pm] TrevorParscal joined the chat room.

[4:00pm] flipzagging joined the chat room.

[4:00pm] kaldari joined the chat room.

[4:00pm] Philippe: So, Danese.... would you like to update us on what's going on in the tech side of the world, and then we'll open up the chat?

[4:00pm] jorm: w0000! here come the monkeys!

[4:01pm] peteforsyth joined the chat room.

[4:01pm] kim_bruning: jorm, there go the monkeys if you don't be quiet and let danese chat

[4:01pm] Philippe: Danese is typing furiously

[4:01pm] Philippe: i can hear her

[4:01pm] danese: Philippe: well, there's a ton of stuff going on. Let's see...We're working hard on the Data Center, we've been doing some pilot deployments of "new" and "not so new" features. I've been reorganizing the team and setting up ways to manage more humanely. We're getting ready for the Fundraiser. We're hiring like crazy right now...and of course I'm finally smart enough for Office Hours

[4:02pm] Philippe: "humane" management? That's ... untechnical, isn't it?

[4:02pm] Philippe: What's that mean?

[4:02pm] thedj: everyone gets a pet ?

[4:02pm] Philippe: monkeys, evidently.

[4:02pm] kim_bruning: danese, WMF tends to either give people a profound IQ boost, or they fall by the wayside

[4:03pm] kim_bruning: Philippe, I thought CTO's were known to be cat-herders?

[4:03pm] brion joined the chat room.

[4:03pm] kim_bruning: danese, what kinds of people are you hiring at the moment?

[4:03pm] kim_bruning: hello brion!

[4:03pm] danese: Well, there were a few places when I got to WMF where folks had either been understaffed for a loonnggg time, or where a very long duration project (*cough Vector") had taken a whole group of people through a painful waterfall development process...there was a lot of burnout

[4:03pm] Philippe: That's a fair approximation, as far as I can tell, but there's another one

[4:03pm] • brion waves

[4:03pm] aude: hi brion!

[4:04pm] danese: we're working hard to come up with ways that people working for WMF Engineering can more quickly complete and deploy things (get a sense of completion) and we're hiring this year mainly in areas where we have single points of failure

[4:04pm] Prodego joined the chat room.

[4:04pm] thedj: danese: so are you looking into ways to prevent such a thing from occuring again ?

[4:04pm] thedj: someone is reading minds...

[4:04pm] danese: thedj: yes

[4:05pm] kim_bruning: danese, vector went waterfall?

[4:05pm] danese: This year (as I said in my long email a couple of weeks ago) we are hiring in Ops, in General Engineering (things Feature Teams need) and in Volunteer Outreach

[4:05pm] kim_bruning: oh... $@$@@!

[4:05pm] kim_bruning: danese, Ops? Sysops?

[4:05pm] Philippe: So would you bring us up to date on the data center, as well>?

[4:06pm] kim_bruning: danese, not all wikimedians read all emails

[4:06pm] peteforsyth: Question: Danese, you have a pretty extensive background with various volunteer/open source communities. But I'm not sure I've ever heard an overview, and I suspect others are interested in your background too..can you list some of them?

[4:06pm] danese: yes, my observation is that everybody working on Vector pretty much worked on that alone for about 18 months

[4:06pm] guillom: kim_bruning, operations, infrastructure, datacenter, etc.

[4:06pm] danese: that's a pretty long dev cycle

[4:06pm] kim_bruning: guillom, Operations. Gottit

[4:06pm] danese: Okay, Data Center, then Volunteers

[4:06pm] kim_bruning: danese, are you ops-focussed? or are you also looking at devs?

[4:06pm] danese: Data Center: we have been in Tampa, FL since the beginning of time

[4:06pm] danese: WikiTime anyway

[4:07pm] brion: :[re: vector -- original intention was to get things out in stages relatively frequently, but it ended up kinda bunching into a giant release towards the end]

[4:07pm] siebrand: Background on Danese Cooper: http://en.wikipedia.org/wiki/Danese_Cooper, http://www.linkedin.com/in/danesecooper

[4:07pm] kim_bruning: danese, that's pretty harsh criticism of the vector team, from my perspective.

[4:07pm] danese: Its not the best facility in the world (converted office building...very tall...in a hurricane-prone area)

[4:07pm] kim_bruning: siebrand, thanks!

[4:07pm] danese: and more importantly its not in a prime market so there's no way to set up good peering

[4:07pm] danese: we're about to close on space in the Ashburn, VA area

[4:08pm] peteforsyth: siebrand: very helpful, thanks. but I'm hoping for a little commentary from her too

[4:08pm] danese: which is an expensive area for hosting...but all of the folks in the top 10 host there

[4:08pm] danese: and we're negotiating a pretty great discount as an in-kind donation

[4:08pm] danese: meanwhile, we're also negotiating with hardware companies to give us machines

[4:09pm] siebrand: Any discussion leader I could privmsg some questions to? It's 1AM here and I have an long day ahead tomorrow. Have to go, but would like to leave some questions.

[4:09pm] kim_bruning: danese, why is the Ashburn, VA area useful?

[4:09pm] Philippe: Good. More free stuff. I like that, as the guy in charge of the fundraiser.

[4:09pm] aude: danese: will there be any ops staff in Ashburn?

[4:09pm] Philippe: siebrand: You could send them my way

[4:09pm] kim_bruning: siebrand, it's quiet enough that you can ask directly @danese if you like

[4:09pm] danese: as I said a few weeks ago...we have the money set aside to pay for all of this...and now I'm working to make it cheap (as in beer)

[4:09pm] danese: aude: yes, Rob Halsell will be moving htere

[4:09pm] danese: there

[4:09pm] tomaszf: kim_bruning: it allows us to peer with far more networks then we could in tampa

[4:09pm] guillom: kim_bruning, because of the excellent connectivity, afaik

[4:10pm] kim_bruning: tomaszf, guillom thanks!

[4:10pm] danese: kim_bruning: its not uncommon for ambitious projects to become waterfall

[4:10pm] danese: and it shouldn't be seen as a criticism of Naoko or her team

[4:10pm] kim_bruning: danese, fair enough

[4:10pm] tomaszf: kim_bruning: http://en.wikipedia.org/wiki/Peering

[4:10pm] danese: it was kind of baked into the grant, in fact

[4:10pm] thedj: and you have to remember that the vector team didn't really have a structure to build upon.

[4:11pm] danese: that's one reason we're trying not to take restricted grants for Eng projects in the future

[4:11pm] kim_bruning: danese, ah, shoot. That makes total sense to me, and that is a very nice way to put it!

[4:11pm] guillom: (yay)

[4:11pm] HaeB: kim_bruning: naoko komura said "Often times investing in future opportunities was postponed due to immediate problems. " http://blog.wikimedia.org/blog/2010/06/21/wikidashboard-revisited/

[4:11pm] Philippe: So we have an open question from Pete....

[4:11pm] Philippe: background

[4:11pm] • Risker the non-techie in the room wonders what special meaning "waterfall" has in techie lingo

[4:11pm] danese: PeteForsythe: You want to know about my "past"

[4:11pm] StevenW: Risker: :[:[Waterfall model]]

[4:11pm] Philippe: Thanks, StevenW

[4:12pm] TrevorParscal: waterfall would mean one person passes the work onto the next and basically is done with their portion

[4:12pm] danese: So...I've been working on Free and Open Source software since 1999

[4:12pm] kaldari left the chat room. (Remote host closed the connection)

[4:12pm] TrevorParscal: as opposed to extreme programming - short sprints

[4:12pm] brion: Risker: http://en.wikipedia.org/wiki/Waterfall_model

[4:12pm] kaldari joined the chat room.

[4:12pm] brion: (and more fun: http://c2.com/cgi-bin/wiki?WaterFall )

[4:12pm] Risker: thanks folks!

[4:12pm] kim_bruning: TrevorParscal, not so much one person to the next, as one phase to the next

[4:12pm] alolita1 joined the chat room.

[4:13pm] danese: I worked for 6 years at Sun, 4 at Intel (on Open Source)

[4:13pm] TrevorParscal: kim_bruning: fair enough

[4:13pm] danese: and then did a year in Startup land

[4:13pm] Philippe: So I would assume that the open source parts of those were heavily community/volunteer oriented?

[4:13pm] Philippe: (which I think was what Pete was sort of looking for)

[4:13pm] kim_bruning: TrevorParscal, and sprints of ~1 week length are more characteristic of scrum. Xp tends to have shorter-length cycles with no specific name that I'm aware of

[4:13pm] flipzagging left the chat room. (Remote host closed the connection)

[4:14pm] kim_bruning: TrevorParscal, (I currently work at an ICT architecture division of a bank... ^^;;)

[4:14pm] Jamesofur|laptop joined the chat room.

[4:14pm] danese: While at Sun I was involved in the creation of: OpenOffice, NetBeans, SunGridEngine, OpenSolaris, OpenJava, Java.net, Blogs.sun.com and a bunch of other stuff

[4:14pm] flipzagging joined the chat room.

[4:14pm] danese: I've also been on the OSI Board for 10 years

[4:14pm] Philippe: OSI = Open Source Initiative?

[4:14pm] peteforsyth: Yes, this is exactly what I was after -- thx for the prod Philippe !

[4:14pm] danese: yup

[4:14pm] Philippe: Great.

[4:14pm] kaldari: danese = Open Source rockstar

[4:14pm] thedj: 10 years in software is long !

[4:14pm] danese: I'm also an Apache Member

[4:14pm] guillom: diva, not rockstar

[4:15pm] danese: and a Mozilla Advisory Board member

[4:15pm] ragesoss_away left the chat room. (Quit: ragesoss_away)

[4:15pm] thedj: busy bee

[4:15pm] Philippe: So another question from another user: There have been some long discussions on wikitech-l about "Wikimedia vs. community" and "the state (of absence?) of code review" and "the future of MediaWiki releases" most recently. Where do you stand on community contributions to MediaWiki and integrating them in the code deployed by Wikimedia. If you might think that code review would get up to speed again, how long would you estimate

[4:15pm] Philippe: that will take (in months)? What is your position on regular releases of MediaWiki?

[4:16pm] danese: somebody is looking for a pointer to the email I wrote at the time

[4:16pm] • jeremyb has a followup q for that

[4:16pm] kim_bruning: code review and mediawiki release schedule is a very core thing

[4:16pm] danese: but basically many of my plans for this year are Volunteer Outreach related

[4:16pm] Amgine: <could spend a few months following up that>

[4:16pm] jorm: danese's comments on community/wmf development: http://lists.wikimedia.org/pipermail/wikitech-l/2010-September/049207.html

[4:17pm] danese: Anybody got another topic while those who haven't yet read up?

[4:17pm] Philippe: And another submitted privately: End of October there will be a Hack-A-Ton in the US. Great initiative, I think. Aside from the announcement, not too much information has been given about it. Few questions: can volunteer developers make funding requests, and if so, to where? When do you consider the Hack-A-Ton to be a success?

[4:17pm] danese: its a long post

[4:17pm] danese: We're getting ready to announce financial assistance program

[4:17pm] Amgine: QUESTION: WMF is currently building yet another ad server for the 2010 Fundraiser. Every year I ask if we will move to any of the freely-licensed ad servers which are available off-the-shelf and are feature-rich. So, to keep with tradition, will WMF consider using mature tools rather than homegrown with bailing wire and spit?

[4:18pm] aude: danese: any updates on the hackathon? venue set? if not, please pm or e-mail me

[4:18pm] kim_bruning: Amgine,

[4:18pm] danese: I'm hoping the HackATon will be very productive and that we'll be able to do one or more a year around the world

[4:18pm] tomaszf: Amgine: i'm really curious to start looking at open ad server

[4:18pm] jeremyb: Amgine: which are you referring to as moving to?

[4:18pm] kim_bruning: Amgine, I think you don't want to distract danese from the previous question

[4:18pm] kim_bruning: Amgine, which would get our bug fixed

[4:18pm] danese: We've been going through the BugBase looking for things to squash, and we have a couple of planned sprints

[4:18pm] Philippe: kim_bruning: hush! it's an open chat

[4:18pm] Philippe: Amgine is welcome to ask at any time

[4:18pm] kim_bruning: Philippe, well of course he is!

[4:18pm] tomaszf: while it wont make it for this year .. i'm eager to look at its feature set and what we could get

[4:18pm] danese: this first one is intentionally small and in a place we've not spent much time before (DC)

[4:18pm] Amgine: OpenX is the descendent of Open Ad Server; but it's hardly the only game in town.

[4:19pm] peteforsyth: danese: are the planned sprints announced someplace?

[4:19pm] Amgine: I found about 200 in the past hour.

[4:19pm] kim_bruning: Philippe, I was making a suggestion based on what I happen to know of his priorities

[4:19pm] Philippe:

[4:19pm] danese: Peteforsythe: they will be on the wiki page Chad Horohoe started

[4:19pm] Philippe: someone have a link to that page for Pete?

[4:19pm] Philippe: So what's next?

[4:19pm] HaeB: danese: What is the best general advice for a Wikimedia project that wants a particular configuration changed (or a feature developed) for their wiki?

[4:20pm] Philippe: burning questions for the Open Source Diva?

[4:20pm] HaeB: (this one is recycled from http://en.wikipedia.org/wiki/Wikipedia:Wikipedia_Signpost/Newsroom/Interviews_desk/CTO

[4:20pm] jorm: hack-a-ton dc: http://www.mediawiki.org/wiki/Hack-A-Ton_DC

[4:20pm] peteforsyth: HaeB: great q, +1

[4:20pm] tomaszf: Amgine: but the core issues is scale. we really have to spend a good amount of time assessing a tool to find if it can scale to wikimedia project traffic

[4:20pm] Amgine: Danese: Barbara Walker's ssk, or traditional ssk?

[4:20pm] danese: HaeB: Well, if you need a config change, you should file a Bugzilla request

[4:21pm] danese: Features? That's a longer conversation

[4:21pm] Philippe: I recommend bribery. Danese responds to chocolate.

[4:21pm] kim_bruning: I'd really like to discuss priorities with danese at some point. Mediawiki-the-open-source project doesn't *appear* to be entirely healthy

[4:21pm] Amgine: tomaszf: OpenX is already dealing with larger loads than wikipedia impressions.

[4:21pm] kim_bruning: previously due to brion-does-not-scale

[4:21pm] kim_bruning: does danese scale?

[4:21pm] kim_bruning:

[4:21pm] notafish is now known as delphine.

[4:21pm] danese: In general, we're looking at ways to make our Feature Engineering more agile, and we're working on ways for the volunteer community to be more involved in scratching its own itch

[4:21pm] guillom: danese hired EPMs (engineering programs managers) to scale

[4:22pm] HaeB: danese: ok, what is the best general advice for a Wikimedia project that wants a particular bugzilla request to be resolved?

[4:22pm] guillom: kim_bruning ^

[4:22pm] danese: kim_bruning: yes, MediaWiki needs help

[4:22pm] kim_bruning: guillom, COOL

[4:22pm] danese: Its true, I'm very dependent on the EPM model

[4:22pm] Amgine: HaeB: is that a non-Wikipedia Wikimedia project?

[4:22pm] danese: the idea is that everybody works for me, but the EPMs do the day to day resource and time management

[4:22pm] danese: humanely

[4:23pm] danese: HaeB: The first thing is...we're hiring a Bugmeister

[4:23pm] danese: I know you've heard that before

[4:23pm] thedj: yeah bugmeister !

[4:23pm] kim_bruning: danese, are EPMs the best tool for the job when dealing with an open source community?

[4:23pm] danese: but this time I have plans to get one that isn't so over-qualified that they're bored with pruning bugs

[4:23pm] kim_bruning: (or are these ones?)

[4:23pm] HaeB: Amgine: it's a deliberately general question (intended to reflect the interest of signpost readers), not intended to push any particular example

[4:24pm] danese: kim_bruning: EPMs are a good tool for dealing with engineering teams

[4:24pm] arjunaraoc: WMF plans to increase the video content. What is the status on video editing/streaming support infrastructure and plans?

[4:24pm] jeremyb: description of bugmeister?

[4:24pm] guillom: jeremyb, bugzilla guy

[4:24pm] guillom: or gal

[4:24pm] danese: Posting of Bugmeister position should be soon (this month)

[4:24pm] thedj: arjunaraoc: a video eidtor/sequencer is almost upon us.

[4:24pm] jeremyb: as in triage?

[4:24pm] danese: Our HR guy is like a one-armed paper hanger right now

[4:24pm] jeremyb: and is that permanent?

[4:25pm] kim_bruning: danese, I guess I'm wondering whether these guys are pull or push

[4:25pm] kim_bruning: ie. patch integrators and tiebreakers

[4:25pm] danese: arjunaraoc: Video...yes Kaltura has been working for some time on Video tools

[4:25pm] kim_bruning: or if they hire people, pay them to do work, and etc...

[4:25pm] danese: we're about to release a sequencer for folks to play with

[4:25pm] danese: these are Labs projects

[4:25pm] kim_bruning: (open source vs paid development)

[4:25pm] arjunaraoc: thedj: can you elaborate or post a link

[4:25pm] guillom: guys please try not to mix topics, it's confusing and hard to follow

[4:26pm] kim_bruning: (or at least, open source community, vs paid development)

[4:26pm] thedj: arjunaraoc: http://commons.wikimedia.org/wiki/File:Sequence-Cats.ogv this is a sequenced video.

[4:26pm] Philippe: kim_bruning: I think there's a role for paid development within the open source community, isn't there, historically?

[4:26pm] kim_bruning: Well, we can cool the channel down

[4:26pm] Kingpin13 joined the chat room.

[4:26pm] danese: meaning we're learning about how to make those projects work for MediaWiki...many of Kaltura's plans are very futuristic

[4:26pm] guillom: arjunaraoc, http://commons.wikimedia.org/wiki/Commons:Sequencer

[4:26pm] kim_bruning: Philippe, sure, in fact, erik and I have been pushing for it

[4:26pm] kim_bruning: Philippe, Erik obviously more than me

[4:26pm] danese: kim_bruning: most Apache developers are paid by somebody...true today for Linux Kernel devs too

[4:26pm] thedj: arjunaraoc: this is how it is made: http://commons.wikimedia.org/wiki/Sequence:Cats

[4:26pm] kim_bruning: danese, that's absolutely perfect, fine and dandy

[4:26pm] kim_bruning: danese, I tend to get paid too.

[4:27pm] Philippe: OK, so as far as I can tell, the queue is clear

[4:27pm] kim_bruning: danese, but there's no point in making patches (paid or unpaid) , if they don't end up on WP

[4:27pm] Philippe: who's next to punch at Danese?

[4:27pm] Philippe: ah, shoulda seen

[4:27pm] Philippe:

[4:27pm] danese: kim_bruning: you're asking a Code Review question

[4:27pm] danese: So...here's the deal there

[4:27pm] kim_bruning: danese, essentially!

[4:27pm] • kim_bruning listens carefully

[4:27pm] danese: Historically (I mean a long time ago now) anybody could get a patch in (more or less)

[4:27pm] • delphine punches, but just for the heck of it

[4:27pm] BloodRedSandman left the chat room.

[4:28pm] • guillom embrasse delphine.

[4:28pm] Philippe: careful, delphine, i think a match b/w you and Danese is a fair fight

[4:28pm] danese: but that led to some really crufty areas of MW codebase

[4:28pm] • delphine embrasse guillom

[4:28pm] danese: delphine: :-*

[4:28pm] delphine: Philippe: héhé

[4:28pm] kim_bruning: danese, so what's the solution now?

[4:28pm] danese: Then Brion and Tim Starling decided to take on the job of reviewing code that made it into core

[4:28pm] danese: then Brion left

[4:29pm] kim_bruning: danese, also Brion-Does-Not-Scale

[4:29pm] danese: and poor TimS has been working hard to keep up with a ridiculous queue

[4:29pm] • kim_bruning listens

[4:29pm] danese: yes

[4:29pm] kim_bruning: and TimS-does-not-scale

[4:29pm] danese: *and* TimS is about to take some time off

[4:29pm] kim_bruning: zomg

[4:29pm] danese: because he's about to become a Dad

[4:29pm] delphine: baby!

[4:29pm] kim_bruning: we are doomed!

[4:29pm] danese: I know

[4:29pm] thedj: oh right. kid on the way

[4:29pm] Philippe: code freeze!

[4:29pm] danese: So of course we started working on this problem

[4:29pm] Kingpin13 left the chat room. (Quit: You're a kitty!)

[4:30pm] guillom: another wikibaby

[4:30pm] danese: for awhile we were growing other resources, one especially

[4:30pm] • kim_bruning is now REALLY curious about the answer.

[4:30pm] • kim_bruning listens

[4:30pm] danese: but that guy recently had an accident...and that is a problem

[4:30pm] danese: Here's the good news:

[4:31pm] danese: I think we're going to have to look hard at how to invest code review privs in many more folks

[4:31pm] jeremyb: as in car colision?

[4:31pm] danese: probably from the Community

[4:31pm] jeremyb: collision*

[4:31pm] jorm: jeremyb: something like that.

[4:31pm] danese: jeremyb: bike collision...he's been in hospital ICU for more than a week

[4:31pm] jeremyb: damn

[4:31pm] kim_bruning: (we've had bus-problems too? )

[4:31pm] Philippe: So, danese, you're talking primarily about using community resources for that? or staff? or some combiination of both?

[4:31pm] danese: So...in my past experience...

[4:31pm] Krinkle left the chat room.

[4:31pm] kim_bruning: danese, go on!

[4:32pm] Panyd joined the chat room.

[4:32pm] kim_bruning: danese, I think this is probably seen as the biggest challenge for you.

[4:32pm] • JeanFred notices the capital C to Community

[4:32pm] danese: When a project needs to solve resource problems and they can't hire (for whatever reason) those resources...it ends up being a forcing function for a return to more community involvement

[4:32pm] Philippe: JeanFred: I wouldn't read too much into anything I type. It's rarely with forethought

[4:32pm] StevenW left the chat room. (Quit: StevenW)

[4:32pm] • kim_bruning wonders if danese is thinking of a torvalds model (with lieutenants etc...)

[4:32pm] danese: JeanFred: you're right, should have done a C there

[4:33pm] danese: Actually that's what we were doing before (Torvalds)

[4:33pm] danese: I'm thinking more broadly

[4:33pm] danese: like Apache

[4:33pm] kim_bruning: danese, the community is trying to push their patches forward desperately. But no one is picking them up.

[4:33pm] kim_bruning: danese, this is hurting several wikis.

[4:33pm] danese: anybody who can demonstrate aptitude for the job can get "Code Review" rights

[4:33pm] kim_bruning: danese, you could do without paid dev entirely (not saying you SHOULD but)

[4:34pm] danese: kim_bruning: actually, I disagree with that

[4:34pm] kim_bruning: danese, but if you can't handle community patches, you/WMF will be in trouble

[4:34pm] kim_bruning: the latter with some level of certainty

[4:34pm] danese: We're trying to push the uptime from around 99% to 99.999% over the next five years

[4:34pm] Risker: I disagree, kim_bruning, because we have seen too many volunteers wander off halfway through tasks they have taken on

[4:34pm] danese: you get there with paid staff

[4:34pm] kim_bruning: danese, uptime is a good thing to have!

[4:35pm] jeremyb: so, we're at 2 9's now?

[4:35pm] thedj: so we are gonna asign reviewers by agreement.

[4:35pm] danese: But I agree that much of our development should be more of a partnership

[4:35pm] kim_bruning: Risker, sure, on the other hand, if patches actually DO get delivered, they are consequently not integrated

[4:35pm] adam_miller left the chat room. (Quit: adam_miller)

[4:35pm] danese: there are still not going to be many volunteers who want to do the really nasty or boring jobs

[4:35pm] kim_bruning: this is true

[4:36pm] geniice: jeremyb 99% allows 3.65 days of downtime a year

[4:36pm] danese: kim_bruning: I get it that patch writers are feeling pain of existing Code Review

[4:36pm] kim_bruning: danese, yes, and some of those patches are fairly essential to continued viability of certain projects

[4:36pm] danese: and I want those patches, I really do

[4:36pm] thedj: well and then there is the experience issue. A good review takes a lot of experience.

[4:36pm] Philippe: Is it important to define downtime in this context? As I understand it, we use a very aggressive definition that inclues more than just "site up" or down, but also includes internal systems, etc.

[4:36pm] kim_bruning: at least one of them was. I think we got a temp workaround? <looks at amgine>

[4:36pm] Philippe: Is that correct, danese?

[4:36pm] danese: yes...our community of Code Reviewers are gonna gain a lot of experience over the next couple of years

[4:37pm] geniice: 99.999 is 8.76 hours of downtime a year

[4:37pm] Betacommand: danese: Something that I have noticed since brion stepped down, a lack of communication and response from the techs when something goes wrong, example, on the 16th the API servers where giving 504 errors, it took the dev team 2.5 hours to figure out they had a typo in CommonSettings all the while they where saying it was something not related

[4:37pm] danese: Philippe: yes, that's correct

[4:37pm] guillom: by the way, I don't think this has been linked yet: http://www.mediawiki.org/wiki/Code_review_guide

[4:37pm] kim_bruning: geniice, doesn't tandem advertise that?

[4:38pm] danese: Betacommand: Well, we've been working on communication

[4:38pm] brion: Betacommand: in all honesty that kind of stuff has always happened

[4:38pm] danese: but first of all, I want you to appreciate what actually happens in an outage right now

[4:38pm] brion: not every problem source is discovered immediately

[4:38pm] Amgine: I'm sorry, let me catch up.

[4:38pm] brion: sometimes debugging goes down the wrong path for a while

[4:38pm] danese: There are a very small group of guys who can deal with an outage

[4:38pm] guillom: Betacommand, I actually think we've been improving communication about tech issues. It's also a timezone problem, and one of the reasons for building a more geographically-distributed operations team

[4:38pm] Bdka joined the chat room.

[4:38pm] Dragonfly6-7 joined the chat room.

[4:39pm] danese: typically they get woken up...they fight the fire...and then they figure out why it happened

[4:39pm] danese: they do this as quickly as they can

[4:39pm] Amgine: Danese: There are literally hundreds of patches which are solely waiting on the few people allowed to commit them.

[4:39pm] HaeB: geniice: not, that would be 99.9 %. http://en.wikipedia.org/wiki/Five_nines

[4:39pm] Panyd left the chat room. (Ping timeout: 265 seconds)

[4:39pm] danese: Amgine: I think we've already talked about the patch situation

[4:39pm] flipzagging: Folks, here's a picture of Danese answering your questions right now: http://commons.wikimedia.org/wiki/File:Danese_cooper_irc_office_hrs_2010_09_22.JPG

[4:39pm] Bdk:[a] left the chat room. (Ping timeout: 252 seconds)

[4:39pm] kim_bruning: danese, okay, I appreciate we can't take all your time on one topic now.... I'd really love to discuss the code review situation in more detail later

[4:40pm] aude: danese: can you tell us more about the "data summit" mentioned in the july 2010 wmf report to the board

[4:40pm] thedj: actually one more thing on communication

[4:40pm] danese: Often they can't peel themselves away to "communicate" about a given problem

[4:40pm] HaeB: aude: +1

[4:40pm] • jorm luckily cropped from that photo.

[4:40pm] danese: but we're trying to change that

[4:40pm] jeremyb: flipzagging: guillom behind here?

[4:40pm] jeremyb: her*

[4:40pm] guillom: jeremyb, no, tomaszf

[4:40pm] jorm: that's tomaszf

[4:40pm] tomaszf:

[4:40pm] flipzagging: Oh sorry, I should have noted that

[4:40pm] thedj: i suggested today that like OFFICE actions, it would be a good policy if sysadmins mark their editsummaries with a link to a sysadmin policy or something.

[4:40pm] Amgine: Danese: well allow me one caveat: within those awaiting review are nearly 100% of non-Wikipedia-submitted patches.

[4:41pm] guillom: jeremyb, :[:[foundation:Staff]]

[4:41pm] Amgine: If it isn't for WP, it's second class.

[4:41pm] KFP: flipzagging: *Gasp* Macs. Evil.

[4:41pm] Risker: danese, in a different line of thought now...one of the challenges large communities have is developing a consistent path of communication with the "tech" side of the shop. Have you given consideration to tasking someone to acting as something of a clearinghouse for things like that?

[4:41pm] GerardM- joined the chat room.

[4:41pm] jeremyb: guillom: i thought the interwiki was wmf

[4:41pm] Philippe: Great question, Risker

[4:41pm] kim_bruning: Good question, Risker

[4:42pm] GerardM-: sad for being late

[4:42pm] danese: The hiring plan for this year includes a Volunteer Engineering Coordinator

[4:42pm] danese: who will work on that direct communication

[4:42pm] Risker: well, simply put, I don't think we should have to bother Danese with minor operational issues, so I am really glad to hear about that role

[4:42pm] danese: I've also contracted a Tech Writer (who I hope will become an employee) with a huge experience in Community interaction

[4:42pm] kim_bruning: hello GerardM-

[4:43pm] danese: between them I expect communication to get much much better

[4:43pm] kim_bruning: GerardM-, your turn to be late for once!

[4:43pm] kim_bruning: danese, and this communication will be two-way, right?

[4:43pm] thedj: i personally always try to connect the dots between english wikipedia, developers and wikitech IRC, but time is limited of course

[4:43pm] danese: And now that things are starting to change, I'll have time for typing myself

[4:43pm] danese: and I plan to blog a lot more

[4:43pm] • kim_bruning triplechecks

[4:43pm] Philippe: no, kim_bruning, we thought we'd set up some sort of soundproof barrier.

[4:43pm] thedj: rofl

[4:44pm] Philippe: Danese: data summit?

[4:44pm] danese: Okay, the Data Summit...

[4:44pm] GerardM-: how about supporting scripts that are problematic like Devanagari (Hindi among others) ?

[4:44pm] • jeremyb has heard of one way mirrors but not one way sound proofing

[4:44pm] kim_bruning: Philippe, I tend to stick to one wiki-location a lot, and from my perspective, sometimes that's what it looks like ^^;;

[4:44pm] danese: Original plan (and I'm still trying to make this happen, but we're about to run out of time for PlanA) was to host a small meeting for folks interested in Data topics in MediaWiki

[4:45pm] kim_bruning: Philippe, WMF provides services to communities, so it had best listen. (worst case a community can fork)

[4:45pm] • kim_bruning nudges GerardM- Dude, DATA!

[4:45pm] danese: But now I think we may have to punt it to November

[4:46pm] privatemusings joined the chat room.

[4:46pm] Philippe: Just as a time check... we have about 14 minutes left

[4:46pm] aude: danese: by "data topics" you mean data as content (e.g. semantic, map data, & other types of data) rather than site analytics

[4:46pm] Risker: One more question...any thought of WMF hosting an equivalent of Toolserver in some country with less restrictive privacy rules?

[4:46pm] jeremyb: data and the dc hackathon are unrelated?

[4:46pm] danese: So, amongst the Data topics: Structured Data, Analytics, Research

[4:47pm] jorm: they are unrelated.

[4:47pm] danese: jeremyb: related only in that we were trying to save airfare by only flying some folks once

[4:47pm] kim_bruning: Philippe, This is not the fault of WMF per-se. It's the tendency that any centralized organisation has, so you need to be pro-active to avoid it

[4:47pm] nimish_g joined the chat room.

[4:47pm] HaeB: danese: analytics as in http://www.mediawiki.org/wiki/Analytics_upgrade ?

[4:47pm] pdhanda joined the chat room.

[4:47pm] kim_bruning: Philippe, no one makes a "let's fubar communications with our base" policy. It tends to happen automagically.

[4:47pm] danese: but we've not found a space in DC that would work for both HackATon and that summit

[4:48pm] danese: Risker: Toolserver is something I've been giving a lot of thought to

[4:48pm] danese: the Community obviously loves it

[4:48pm] Philippe: <3

[4:48pm] aude: danese: re DC hackathon venues, please e-mail me asap (i'm signing off in a few minutes)

[4:48pm] kim_bruning: Philippe, (I work at the central services organisation at a collective bank, so I've seen this from both sides now )

[4:48pm] danese: the German Chapter guys were here in SF last month and we had some conversations about where it should live, etc.

[4:49pm] danese: and how to make it better serve Community

[4:49pm] danese: we're not ready to say what we concluded, but talks are ongoing.

[4:49pm] Risker: excellent. The German Chapter has been a fantastic trailblazer in this area

[4:49pm] Philippe: OK, ten minute warning. If you have any final questions, now is a good time

[4:50pm] danese: I'd really like to see the stats on Toolserver better validated

[4:50pm] Amgine: did the ssk question get answered?

[4:50pm] GerardM-: How about script and font support ?

[4:50pm] Philippe: And while we wait, I'm going to plug upcoming office hours: Sue Gardner, Cyn Skyberg, our new Chief Talent and Culture Officer, and Zack Exley, the Chief Community Officer

[4:50pm] Philippe: watch the meta page for info on dates/times, and I'll post to the usual lists as well.

[4:50pm] danese: GerardM: as you know, I'm a huge fan of better serving the Indic languages

[4:51pm] zakg: Hey All: I'm the tech writer that danese mentioned. I'll be doing office hours next Wednesday and Thursday. Visit http://mediawiki.org/wiki/User:Zakgreant for more info.

[4:51pm] geniice: danese musical notation support? or are we just going to have to wait for someone to create a secure version of lillypond?

[4:51pm] GerardM-: I am going to Bangkok ... this is about Burmese script and pedias..

[4:51pm] HaeB: danese: (toolserver) would that involve changes to https://wiki.toolserver.org/view/Rules#Privacy_Policy ?

[4:51pm] danese: ErikM and I are planning a trip to India in December to talk about gaps in current WMF offerings, and I know (from talking to you in Poland) that you know a ton about this too

[4:51pm] Risker: nice to meet you, zakg!

[4:51pm] geniice: LilyPond

[4:51pm] danese: I think you can expect to see work in that area over the next couple of years

[4:51pm] arjunaraoc: GerardM: Unless there are some Mediawiki bugs, I feel that local governments/institutions have to take up a bigger role to improve support for languages on computing platforms

[4:51pm] GerardM-: it works best when that work is supported by WMF

[4:52pm] GerardM-: arjunaraoc that gets us nowhere

[4:52pm] danese: zakg: hey thanks for popping up

[4:52pm] arjunaraoc: GerardM: any instances of past experience?

[4:52pm] kim_bruning: GerardM-, to which level? Just "we are WMF, and we support this message" , or ...

[4:52pm] danese: geniice: musical notation is something TimS keeps asking me about

[4:52pm] kim_bruning: arjunaraoc, I think GerardM- actually attends ISO meetings, or something...

[4:52pm] danese: Its one of the candidate for the HackATon

[4:52pm] • zakg smiles and does his best Queen wave

[4:53pm] thedj: zakg: we all stand in awe

[4:53pm] danese: but of course whether we get to hack that one out in 2 days will depend on who comes

[4:53pm] kim_bruning: danese, there's no patch hiding in between the chaff in bugzilla?

[4:53pm] guillom: zakg, with the hat too? (we want pix)

[4:53pm] Prodego left the chat room. (Ping timeout: 272 seconds)

[4:53pm] aude: good night folks (danese: get well soon)

[4:53pm] • kim_bruning looks innocent

[4:53pm] aude left the chat room. (Quit: Leaving)

[4:53pm] privatemusings: has wmf looked at 'wikifonia' ?

[4:54pm] privatemusings: (for music notation, might be a friendly partner?)

[4:54pm] kim_bruning: privatemusings, is that music notiation, or a new topic?

[4:54pm] geniice: danese>support for .blend or other 3D format?

[4:54pm] kim_bruning: music notation

[4:54pm] guillom: geniice, I know Erik is interested in blender

[4:54pm] geniice: <privatemusings they yse LilyPond

[4:54pm] danese: geniice: well now you're getting into things I'd love to do now, but first we have to fix uptime, etc.

[4:54pm] kim_bruning: guillom, that sounds like Erik alright

[4:54pm] kim_bruning: danese, and patches!

[4:55pm] danese: kim_bruning: yes, and patches, Code Review, volunteer work in general

[4:55pm] geniice: danese why? 1)downtime =money and 2)downtime doesn't appear to be a significant cause of complaints

[4:55pm] kim_bruning: (what kind of open source project doesn't handle its patches? :-))

[4:55pm] Ashlee: "fix uptime"?

[4:55pm] Ashlee: What does that mean?

[4:55pm] danese: geniice: you've not gotten the calls from the BBC, CNN, the White House, etc. when we have a significant outage

[4:55pm] privatemusings: the uptime is causing problems

[4:55pm] kim_bruning: Ashlee, they're at two nines, and want to go to five nines

[4:55pm] KFP: Ashlee: Uptime = when WP works.

[4:56pm] kim_bruning: danese, you're kidding about the white house, right?

[4:56pm] Ashlee: Is there a problem with uptime?

[4:56pm] kim_bruning: right?

[4:56pm] Philippe: Basically, we hate uptime. it's bad.

[4:56pm] Philippe: We want less of that.

[4:56pm] KFP: Ashlee: Not really.

[4:56pm] Risker: Ashlee, there isn't enough of it

[4:56pm] geniice: Ashlee well we lost europe last wek

[4:56pm] danese: kim_bruning: but I'm not promising that we'll accept 100% of patches submitted. No project doest hat

[4:56pm] danese: does that

[4:56pm] Ashlee: KFP: So it'd be difficult to fix it...

[4:56pm] kim_bruning: Uptime is like dakka. You always want moar!

[4:56pm] Ashlee: Risker: Seems up most of the time to me.

[4:56pm] KFP: Ashlee: Well, there was a problem here in Europe last week.

[4:57pm] Philippe: OK, friends, romans, and Ashlee.... i'm going to start to wrap this up.

[4:57pm] Ashlee: Europeans don't count.

[4:57pm] GerardM-: Danese are we going to improve our support for the REALLY big pictures ?

[4:57pm] Philippe: As always, the logs will be posted on meta.

[4:57pm] Philippe: And we need to thank Danese for showing up on a day that she's feeling ill.... I know folks have been wanting to have her for office hours for a while

[4:57pm] kim_bruning: http://tvtropes.org/pmwiki/pmwiki.php/Main/MoreDakka <- (surgeon-general's warning, TV tropes will ruin you)

[4:57pm] Philippe: We'll get her scheduled again at some point in the future.

[4:57pm] danese: GerardM: yes, I'm planning to start a Labs that looks at the future and the big big picture

[4:57pm] GerardM-: WOW

[4:57pm] danese: but that's not in this year's plan

[4:57pm] kim_bruning: danese, you're tougher than I am this flu-season! Get well soon!

[4:57pm] Philippe: Danese, any final comments for us?

[4:57pm] thedj: yes thank you danese, for beeing with us.

[4:58pm] thedj: -e

[4:58pm] danese: kim_bruning; what we need to do is graph how many devs are out as a result of me coming in today

[4:58pm] • jorm just explained what "More Dakka" meant to everyone in the room so that they don't have to click the link.

[4:58pm] danese: This was a lot more fun than I expected...were you guys being easy on me?

[4:58pm] kim_bruning: danese, and get Randall Munroe to draw the comic?

[4:58pm] danese: I could see doing this again

[4:59pm] Philippe: Yes. For instance, Ashlee was gagged for about 45 minutes.

[4:59pm] • TrevorParscal want's his 15 seconds back from jorm's explaination

[4:59pm] kim_bruning: danese, we totally would love to see you do this again!

[4:59pm] privatemusings: thanks danese

[4:59pm] Amgine: <was expecting to be knitting during this meeting. That didn't happen>

[4:59pm] • delphine pokes danese, now the fun really starts.

[4:59pm] KFP: Thankee, danese.

[4:59pm] Risker: thanks, Danese. This was very informative

[4:59pm] Philippe: Thanks all.... we'll see you next time. Same bat-channel and all that.

[4:59pm] guillom: delphine, tu devrais être couchée

[4:59pm] danese: delphine: thanks for listening in

[4:59pm] kim_bruning: danese, hang out on #wikimedia-tech anytime you're in

[4:59pm] privatemusings: did we have every continent represented, btw?

[4:59pm] danese had voice removed by you.

[4:59pm] guillom: holy IRC batman!

[4:59pm] privatemusings: Oz here, for the record......

[4:59pm] kim_bruning: danese, questions will come much slower

[4:59pm] delphine: danese: I thought a bit of female presence would come in handy

[4:59pm] You demoted yourself from operator.

[4:59pm] kim_bruning: danese, also hang out on #mediawiki

[5:00pm] Philippe: *** END LOG ***