AC, Social

Political activists and beta testers (update)

A Race Department discussion just created one single post that is worth thinking of:

yesmen_OP

The context is a bit confusing, but people are discussion about the relationship between the Kunos developers and their betatesters for AC. This post is explicitly not talking about Assetto Corsa, but “betatesting in general”, and the fear of an unhealthy “yes man” mindset.

“Unfortunately” Stefano (Kunos lead developer) answered on the official AC forum before I could write down my thoughts – but I don’t believe him.

yesmen_kunos1.png

Disclaimer: The following is highly subjective and more guess than fact.

It seems that this is a political answer. Which means it contains message that isn’t directly expressed, while telling “a truth” that isn’t necessarily “the truth”.

He explains that the betatesters are just sheep with binary bug reports as communication interface between testers and developers. He even downplays their role further by adding that “whatever happens next” isn’t the concern of a beta tester, and even that a bug might not be fixed. Is it possible that beta testers are actually involved so little?

Well, Kunos beta testing is well protected – we don’t know much about who actually is a beta tester, what they do, when they do it and so on. Of course there is this blog that occasionally claims that AC beta testers leak reader submissions, but this can’t be proven and the overall aggressive and unprofessional stance towards Kunos doesn’t really help with credibility here.

Still, sometimes there are hints from Kunos developers. This one is pretty innocent, but tells us that at least 6S.Manu is considered being part of the beta group, maybe he is even organizing the beta crew. Another thing I unfortunately couldn’t find the source, is the gyro oscillation that came with 1.2. This Tweet indicates Stefano’s excitement about this feature during the 1.2 beta:

But it never really made it. Until today, the exciting “gyro thingy” is just an .ini file option tagged with “Experimental”. I’m 100% sure (link very appreciated) there was the official statement that the reason was the feedback of some beta testers who really didn’t like the effect. Given that, beta testers are just bug report sheep? I don’t think so.

Another point Stefano made himself very clearly:

most betatesting in race sim is voluntary

You want good testers. Car physics, development process, testing strategy, accuracy, endurance, reliability. Good testing is pretty boring. If most are doing this voluntarily, what do they get? How are they motivated to bring effort voluntarily, if they are handled like stupid sheep? The only point I can see is access to a beta that isn’t public yet (in AC the hype for new content is real) and maybe belonging to a special group. No, you can’t tell me this alone is working out and you always have enough motivated and qualified testers.

Another indicator that feedback is part of the beta process:

A last point is that the requirements for helpful beta testers are pretty similar to the one’s described in the “Liquid game designers and rock stars” article. If you already have a closed group of selected guys that are actually qualified to beta test new release – why wouldn’t you listen to them?

—–

Let’s call it a theory: Beta testers are more involved than Stefano wants us to think. Why would he? One reason could be that there is a rebellion going on inside the beta community, and he takes the chance to remember his beta testers where the lowest common factor of testing is. Would be a weird communication channel as he surely has better options, but who knows?

Another possible reason is that he wants to  work against beta leaks and reader submissions. Less involved beta tester mean even less credible sources for leaks (if real or made up).

Disclaimer: Maybe I’m misreading this one and he just was in the mood to disagree and therefore gave an extreme view of how it could be, although this isn’t the case in reality. To be honest this is the most likely conclusion – but the point was worth a thought.

——————————–

Update:

As the discussion is still hot, Stefano gave a motivation to post this I completely missed in the context:

[…]fact remains that devs are under no obligation to follow your feedback nor to provide you with explanations about their choices.

So his extreme post was a reaction to a statement I didn’t (and still don’t) see. Apparently there was a suggestion that developers have to obey to betatester’s feedback – more or less.

I was wrong then, sorry.

2 thoughts on “Political activists and beta testers (update)

  1. Being a beta tester and a dev that has run these sorts of things before albeit on a smaller scale as the largest game ive been involved with made 50,000 sales.

    Firstly 6s.manu is a dev so hes above the beta testers, hes likely the one that fixes the stuff that the testers report.

    I think ive run 5 or 6 beta programs now, each with 40-50 people in them that seem to have a rather heightened interest in the game(sending emails, commenting on the community forums etc etc). I do a background check to make sure they arent a catfish trying to get a free key which they sell on G2A or similar sites.

    You generally have three types of beta testers, there are the typical guys that enjoy the content and dont really test or do anything, always positive. Some refer to them as the mindless sheep but i think that is harsh, these guys tend to be the ones that get mad when things dont go their way.

    Then you have the grinders which will report bug after bug after bug, you generally dont know when these guys get the time or if they sleep, every now and then you get curious and even consider if they are actually human.

    Finally we come to the rarest and my favourite, i call them the golden tickets, these are the guys that devs cherish well at least I do, others may not. They are in their prime for testing and will grind at times but they have great suggestions , sometimes they think about how something can get implemented, they are general friends which help keep morale high. You dont always get them, but they are fantastic because of how rare they are. Of course there is a chance that they could be a bad apple but that has never happened with me, these guys invest a lot of time in helping with improvements.

    Stefano is right though, while there isnt a definitive answer for testing you can optimize and in the case of kunos they have modders which they have used to become devs and likely to help with testing as well.

    Its a shame these sorts of things are never taught, dedicated game dev courses dont cover it, they dont cover the risks, in a large game they dont cover the chance of a hate group forming and how to reduce the chances of a beta tester turning on you

    Like

    1. Thank you. Actually somebody is reading this Oo

      > Firstly 6s.manu is a dev
      Yes, maybe I didn’t make this clear. The link goes to his post where his developer status can be seen, but I probably should have been more precise.

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s