Develop 2014 Notes

The other week I was at the Develop Conference in Brighton, and I took some notes of sessions and other things that I saw. Because I’m nice, I’m going to share them with you all both.

Session Notes

Player Driven Development Phil Mansell (exec. producer for Runescape at Jagex)

  • Thought this was a really interesting talk, especially how much control they give over to players.
  • They really heavily include their players in Runescape’s development now, taking a lot of creative direction. The point of it all being that players get a game that’s more what they want, and because they’ve felt involved in it they’re more engaged & loyal.
  • They view their player involvement in dev as being a major part of Runescape’s success – they have 15k posts per day on their official forums, are usually in the top 10 Twitch streamed games, a conversion rate of 40%, and over half of their players have accounts 6+ years old.
  • There are 10 things they use to get info off their players. A numbered list!
  1. Community Sentiment Reports.
    • These are hand-written 24 hours after every update. And broken down by channel. Each point is given a red/amber/green status regarding the community’s sentiment towards a particular change.
    • Labour-intensive to make, but easily distributed & easy to glance at. Only captures vocal community.
  1. Play testing: Player visits
    • They bring people (clan leaders, community advocates, etc) in to the studio, all expenses paid for by Jagex. They use these for early previews and have long playtest & feedback sessions.
  1. Play testing: Usability
    • They use Player Research and also their in-house lab to record sessions & interview players.
    • Both of the play testing things are used to gauge stuff when there’s still plenty of time to make changes (they’ll do this for speculative changes, or major new systems). But it’s a small sample size.
  1. Surveys: Customer Service
    • Link appended to support emails, users rate their experience.
  1. Surveys: Targeted Questionnaires
    • When they want to deep dive on specific features, or specific player cohorts.
  1. Surveys: Customer Satisfaction
    • General “how much do you like Runescape” surveys, broken down by content & aspect (billing, combat, client performance, CS). Re-polled monthly.
    • All the surveys allow them to see impact of updates & trends over time. But expensive to run regularly. Depending on your game you could also integrate the survey in to your game client.
  1. Automated Chat Analysis
    • Keyword counting in in-game chat, given a +/- bias. Also detects trending words. Then a person goes through and adds commentary (suggesting reasons for particular trending words, such as the world cup). Aside from the commentary it’s automated & emailed out daily, so once it’s in place it’s very low maintenance.
    • The bias is pretty unreliable (no sarcasm), but the general analysis does help highlight unanticipated issues. Trending words hasn’t been useful yet, just interesting.
  1. Predictive Modelling
    • Using analytics to see trends of events that lead to players churning, and also patterns that lead to increased engagement. They then feed these back in to the client, and when it spots a player following a negative model they nudge them with some kind of custom high engagement = reward feature they have). They also use the model to adjust content offline (for example if 75% of people who plays a certain level never come back to the game, they kill that level).
    • This requires a huge strategic investment from the company (in terms of expertise & tech infrastructure), unless you use an external provider (but an external will always be a bit generic & not 100% tailored to your game).
  1.  Customer Relationship Management
    • A system built in to the game that connects all of their communication channels in to one central hub, and key display points through the game. . Means that players don’t have to follow all your channels to find out what’s going on. What players see is segmented to avoid annoying them with stuff they would hate.
    • Another large investment to get this in to your game, but the best way of making all of your players aware of new content etc.
    • How much benefit your game will get depends on your output rate in terms of new content, offers, news, etc.
  1. Player polls
    • They have put a/b polls inside the game asking players what stuff they want to see next. At least one new poll a week.
    • You have to make sure you are either happy giving up creative freedom to the players, or that you don’t ask them about stuff you don’t want to change. Players feel super-involved in the game’s development which is a massive boon. Have to be very open with players, otherwise their expectations of what a poll result might mean could make the whole thing backfire.

Managing the Creative Process for a AAA Production Sebastien Ebacher (producer at Ubi, most recently on the Spies vs Mercs in Splinter Cell Blacklist)

  • I quite liked this. Basically a method for removing some of the fuzziness around the progress on a task (one dev’s “it’s 35% done” isn’t the same as another’s).
  • On the quality/time/budget triangle, at Ubisoft it’s the quality that will make you fail – this is viewed as being most important. So they need to manage quality.
  • Conflicts between design/art/tech are important to push the game to be the best it can be. Since they’re unavoidable, structure these debates to be productive & professional, not shouting matches.
  • They use a production pipeline of 1 month sprints.
    • End of sprint meetings with the leads focussed on scope adjustment & planning.
    • Start of sprint full team meeting & team play session, so everyone can see where the game’s at (on a large AAA team it’s not uncommon for some people to do their work without regularly booting the game).
  • Never miss the meetings or put them off (stuff will often seem more urgent), they’re important for visibility & transparency, which affects team morale.
  • The team wanted status updates to be clear and meaningful – no “this feature is 47% complete”. So they split progress of a feature in to 5 stages.
  • Status stages:
    • Creative Brief: The feature’s meaning & purpose. High level intentions. A ppt deck. Agreed by leads as being something that is developable.
    • Feature Sign Off: The design doc (see below).
    • Prototype: The feature is playable by the team.
    • Alpha: It’s playtestable – QA inside the team can play it without instruction.
    • Beta: It’s reviewable. People outside the team can play it.
  • Feature Sign Off in more detail:
    • An Excel sheet, with each line being an element of the feature.
    • Details expected behaviour by element.
    • The design requirements.
    • This becomes a QA doc for the feature.
    • For each line, also lists what Status Stage is it needed for (vfx, for example, could be for Beta)
  • This makes each stage sign-off binary, for each feature you have a “go / no go” call at the end of the sprint on whether it has progressed to the required stage.
  • If “no go”, leads decide what’s needed to make the feature pass. Or cut the feature.
  • Each milestone has targets – how many features should be at each stage. You can then count up the number of validations.
  • Example: “milestone 1, 10 Creative Briefs”. Team managed 6 validated. So missed 4 = “K/D ratio: 1.5”
  • Production sets validation targets per milestone based on required velocity.
  • Velocity = (Validations * features) / Time Unit.
  • The downside of this method is that it’s very meeting-heavy for leads – lots of validation meetings.

The Future of Free to Play A conversation between Nicholas Lovell (gamesbrief) and Prof. Richard Bartle (inventor of MUDs & Bartle Types amongst other stuff)

  • Because it was a discussion, a lot of it was contradictory. For example, Bartle thinks F2P has a “half-life” (that he admits is likely in the order of 10’s of years) before people get sick of it, whereas f2p-advocate Lovell unsurprisingly disagrees.
  • In future years, developers will have grown up with f2p being a thing (rather than current devs, who have all had to adapt), so there could be less vocal resistance to it.
  • Methods will evolve & adapt – whereas currently lots are emulating the MTX techniques of Clash of Clans & Candy Crush, it’s reasonable not to expect these to last forever, people might “have their fill” and want fresh stuff.
  • Players will be more used to it too – more likely to have seen it before & recognise the “pay points”.
  • Bartle suggested there is a fixed number of whales, and though Lovell didn’t pull him up on it, this seemed a bit wrong-headed to me, since it suggests a conclusion that some people are unwilling to spend on anything, ever.
  • Bartle’s takes on his own player types doesn’t seem to match how they’re being used in the wild – but they’re still useful for taking a more rounded approach to satisfying players by ensuring you have things in your game that will hit each base.
  • They agreed that you shouldn’t “pollute” the central concept of the game with MTX – don’t force people to spend to get core features, see things around that.

The Business of Designing Games A discussion panel of Jason Avent (Boss Alien), Simon Oliver (Hand Circus), Matthew Wiggins (Jiggery Pokery).

  • Some guys chat about what they think makes for successful (in a financial sense) games design.
  • On CSR Racing they have been focussed on increasing retention, rather than specifically adding stuff for their identified “whales”. In particular they found that multiplayer has been their “evergreen” feature that’s kept people in the game (before they added that they had fairly high churn).
  • There was an anecdote about a shop that sells sledges, and how that’s only a good business when people have a desire to go sledging (i.e. it’s snowing). The point being that as game developers we can create the desires first (we make it snow), then sell to those desires (we sell the players sledges).
  • Procedural content is evergreen. But the challenge then moves from pure content creation, to how to make it interesting rather than impersonal. A mix of procedural & hand crafted mixed together seems a decent solution (either procedurally combining hand-authored chunks, like Spelunky, or just having some content be one, some the other). Hand craft “just enough” that it feels right.
  • There was some talk of the conflict between creativity and financial goals in a company, where designers want to make stuff because it’s something they personally want to exist, but this stuff could well be a financial dead end. They see the ideal being not just love of a specific product, but people who love creating things other people love, no matter what they are (they mentioned Apple & Pixar as examples).
  • Finally they talked about User Acquisition. Generally this was seen as a patch on top of a game that people didn’t want to talk about – if people wanted to talk about it, then you wouldn’t need to heavily advertise (don’t really agree with this myself, though there are obvious examples of stuff that has succeeded through pure force of word of mouth).
  • Social sharing in games is a way to encourage people to talk about your game, but it needs to be implemented in a way that benefits both players, and preferably without specific rewards for sharing (“tweet this and we’ll give you 5 gems”) because it’s clunky. For example, if a game wants me to tweet a message about it, the people reading that tweet don’t get anything out of the share – they’ve just been marketed to. The Candy Crush life gifting system is an example of the sort of sharing where everyone wins.

Remote Work the Indie Way Eduardo Jimenez from Eclipse Games

  • Essentially the answer was “Communication by whatever medium works for each team member”.

Failing and Learning to Be an Indie

  • Mike Bithell gave a talk about how all of the “first game indie success” stories you know about are bullshit, with the exception of Vlambeer. Nothing groundbreaking, but nice that the indie narrative set up by Indie Game the Movie & years of survivor-biased articles is being challenged a bit these days (there were a few talks along these lines).

Project Morpheus

  • I had a go on Sony’s VR thingy.
  • Played “The Deep” demo. Stuck in a diving cage while Bad Things happen. Felt like a Universal Studios-style fairground ride (but I wasn’t going to play Street Luge the day after a free bar).
  • Higher res than Oculus kit I’ve used (though that was a year ago), but narrower FoV I think. Never felt motion sickness like I have in the past with Oculus.
  • Definitely the immersion is there – I took an involuntary step backwards to “avoid” falling out of the cage.
  • Having AAA teams working on your demo software definitely helps sell it.
  • Still not convinced it’ll ever take off as mass consumer tech – no matter how small & acceptable they make the tech, it is by its nature hugely isolating. You couldn’t be playing it & interacting with someone else in the same room. Can see it doing well with hardcore gamers in their bedrooms though. (And also gamers watching hardcore in their bedrooms.)

Other Games I Saw That I Can Remember

  • So Many Oculus games. Felt like everybody in the world was showing something off. Nothing that felt like it would have any real interest without the VR aspect though. Some people combining VR with other stuff, like haptic controllers.
  • “Ready Steady Play” – a cowboy game compilation that looks lovely as well (the animations are really slick). The basic gunfighting game was super-addictive, I could see an ad supported thing of this gameplay taking off big style.
  • 4PM. A game about being an alcoholic. I could not relate to this one bit. At all. Definitely not. Some bits reminded me of The Prodigy’s Smack My Bitch Up video.
  • A Good Snowman Is Hard To Build. Sokoban, but dressed up really charmingly.
  • Musclecat Showdown. Simple “copy what you see” mechanics dressed up with funny presentation & custom controllers.
  • SwapQuest. Didn’t get to play this, but it seems like a really neat idea. I like things that are entirely obvious from their screenshots. It won the Indie Showcase too.
  • Super Pole Riders. Not really new news, but I’d never got to play it before.