Do we want sustainability to be an evidence based stack exchange site?

Question

A recent question got me wondering what stance we want to take on the evidence requirements for answers (or indeed questions) on Sustainability.

At the moments, as all stack exchange sites say, we expect answers to be supported by facts, references, or specific expertise but different sites have different standards to which they hold answers. On Skeptics for instance all users are required to reference all significant claims they make in their answers.

Most sites do not hold answers to this standard though and just expect the minimum standards laid out in the blog article Good Subjective, Bad Subjective, which promotes the back-it-up principle:

Back It Up! means that your answers must be based on either:

• Something that happened to you personally
• Something you can back up with a reference

They talk about how “opinion, by itself, is noise.” They’re not saying that subjective opinions are to be avoided; they’re attempting to mold and shape their inherently subjective Q&A into something constructive, informative and helpful.

Given the potential for for argument and speculation with a subject as emotive (and opinionated) as sustainability, I think it is worth asking whether we want to hold our answers to this higher standard too.

---

As an additional data point, over on Skeptics Meta I asked How and when did Skeptics introduce its requirement for all claims to be referenced? and the reply there by one of the Skeptics moderators said

Skeptics had this from the very beginning - i.e. private beta.

...

It would have been very hard to enforce such a policy afterwards because of the sheer amount of unreferenced answers standing as broken windows¹

^{1, For those not familiar with it, see Broken windows theory.}

Answer 1

I think it's worth noting that Skeptics.SE allowed itself to build up a body of readers and contributors over months, before enforcing its referencing rule. So when it started enforcing the rule, it had a lot of momentum, which carried it through. After its introduction, it was a "soft rule" for many months afterwards - there are many surviving answers from throughout 2011 and into 2012 that are unreferenced.

I'd argue that had the referencing rule been enforced earlier on, Skeptics.SE would never have made it out of beta, and would have withered and been closed down.

I think that introducing such a rule here, now, would strangle the site at birth. And introducing it without enforcing it evenly is not at all a satisfactory solution.

So here's an alternative, made up of several components, to be taken together, in the interim.

• We should keep coming back to this question every 3 months or so; each time, the question would be framed as "do we now have enough momentum to survive after introducing a referencing rule?"

• if you have serious doubts about the validity of an answer, either ask for clarification in the comments, or post your own answer with references

• be more generous with the upvotes and bounties on answers which you know to be correct, and which are well referenced.

• don't downvote just for lack of references; do downvote if you know an answer to be incorrect, and can back that up with references - in which case, supply those in an answer of your own.

Remember, when asking for clarifications, that our best expert contributors are also likely to be our busiest with real-life stuff: let's not turn into one of those crowd-sourcing sites where those with the least knowledge but the most spare time dominate contributions.

It's also worth asking what higher bar should be set for questions too, if we are to set a higher bar for answers. Otherwise we risk ending up with lots of unanswered, and unanswerable, questions.

Answer 2

SE already calls for complete answers. Answers that include facts are incomplete unless those facts are backed up by reference. A short answer that says what the answer is, is never enough. If it is a short answer it is more important than ever to explain why the answer is correct. No one is served by lots questions with short answers that say what the solution is but do not explain why that solution is the best and how it is sustainable where the more mainstream alternatives are not.

It is very important that we vote often, only vote up good answers and vote down answers that are incomplete. We get the site that we build and if we build a site on up-voting short answers that do not explain and answers with claims of fact that have no references backing them up, then we will get a site that people will not take seriously as a reference for sustainable living.

If you see an incomplete answer post a comment asking the poster to improve their answer. If you see areas of opportunity for improvement suggest them.

• Could you add a link to some reference that backs up your claim of ABC please?
• Could you please explain why ABC is more sustainable that XYZ
• I think your answer maybe right but could you explain how it is sustainable?

On the other hand if we only reward high quality answers, then people with questions about sustainability will value this site. We can become the go to location for answers to sustainability questions, and this site will make the internet a better place in the process help make the world a better place as well.

UPDATE: I am not talking perfectionist, but we do not want to give everyone a trophy just for playing either. One advantage this site has is that there are not an abundance of other sites out there with great content on the subject. There are some scattered sites but compared to many of the other SE topics we are breaking some virgin ground. Keeping our standards high will only help differentiate us from the crap that is sure to pop up.

I would also remember it is important to be friendly and helpful when commenting. I think you are wrong is not a helpful comment, If you read this (link here) you will see why is. And if you see an answer that needs a link and know where you can find a link to support the answer by all means update the answer. The goal here should be to create great content not to have the highest score.

Answer 3

Backing up answers is important, and references are one way of doing that. I'm really not convinced that they are the 'best' way. I find the quality of Q&A on Skeptics much, much lower than most other sites on the network, for example, Math, RPG, and even Workplace. Requiring answers to back things up is important, and sometimes the only good way to do that is with multiple references to scholarly articles, but I don't think that's usually the case and I don't think a references-only policy is something we want to employ (and Skeptics isn't something we want to be like, in terms of quality of answer).