(Sourced from unsplash.com)
Do you publish in books and journals that you think are best for your work?
While this may come across as a dense question, it’s a live and thorny issue for many scholars who are caught in national ‘research quality’ metrics that rank publications, particularly journals.
@thesiswhisperer commented recently that “[c]lassifying my publications by FOR code makes me look like a person who can’t make up their mind what they want to do”.
The title for this post paraphrases @jod999, who responded wisely with: “Your success says a lot more than your #FoR codes. Just keep doing what you do.”
If you haven’t yet encountered a national research quality exercise, I have two things to say to you:
- Congratulations – you still walk in the light; and
- If you’re hoping to hang in academia for a bit, read on to work out how you might negotiate these research quality systems when they cross your radar.
Research quality exercises are created as standardised, supposedly objective modes of measuring the quality of research being produced by research organisations (and, down the ladder, by individual researchers).
The systems are also constantly embroiled in passionate debate about their viability, accuracy, and scope. Is research output the best way to measure research quality? Dare we talk about research impact? What do citations really measure about a piece of work? How much ‘gaming’ of the system, for its own sake, takes place?
In the current climate of Excellence in Research for Australia (ERA), academics need to allocate Field of Research (FoR) codes when registering their publications (and projects) in an institution’s research database.
The ERA exercise – through a complex formula that includes assessments of publications quality, competitive grant income, prestige elements, etc – ranks universities and disciplines according to these FoR Codes.
While there are currently no direct fiscal consequences attached to these rankings, there are many, MANY indirect ones.
For example, if your university was ranked 5 (well above world standing) in a certain FoR code, typical flow-on effects could include:
- research in this FoR code becoming a university priority (if it wasn’t already), which can often mean more material resources and infrastructure,
- encouragement to publish in a particular set of codes to allow the university to build more quality output for the next ERA assessment submission,
- extra funding for personnel to keep this FoR code sailing well (often appearing as commitments to designated new academic positions, scholarships, or research fellowships), and
- researchers who are working well in that FoR code gaining more profile within the university (as scholars to especially support).
In a sector where many already feel disenfranchised and overlooked, many researchers can feel that ERA priorities eclipse their own areas and devalue their work in other fields (that may not have ranked as highly). This is particularly the case if there are edicts to publish only in specific FoR codes and even journals. The issue of curtailing academic freedom has been raised repeatedly in these situations.
For some, the priorities will work very well for where they’re already heading. Others feel that they have to swap area (or disciplinary) horses mid-stream, which can put a significant productivity hole in their track-record.
Into this fraught mix, I wanted to add one more key thing to consider. Something that may run counter to what your department or faculty might be telling you. This is in ‘what do you want to do with your life?’ kind of territory.
For early career researchers in particular, research trajectories and concentrations post-PhD may still be gathering momentum and gaining depth. Collaborations are maturing. If you’re suddenly finding yourself meeting organisational priorities much more than building your expertise the way you’d like to, it’s good to take a step back and think a few things through.
Such as:
- Are those journals that you’re feeling pressured to publish in really where your work needs to be to build your profile? Top journals are sometimes the most conservative, and their publication timelines can be shocking (and not much good for early career researchers trying to get runs on the board for a CV). Across a few years, do three well-received and cited publications in good-but-not-top journals trump one in a top journal? I certainly don’t advocate publishing for the sake of it, but a strong publication track-record doesn’t have to mean only top journal publications.
- Where do you see yourself making your mark? If it’s not in the areas that you’re being shepherded to publish, try a bit of push-back. If you’re researching and writing well in an emerging area, you may well be the next big hope for a university, so don’t mess with your momentum for a system that, quite honestly, may not stick around.
- Further to the previous point: You’re not going to be at your current institution for the rest of your research life…are you? If you feel like you’re compromising your research self because of the priorities of your current institution, think of where an alternative space might be. Cultivate a track-record that serves you well for where you want to go – skewing your work for another discipline or because of colleagues’ exhortations can work well at times, but it can also take you in directions where the work becomes more of an obligation than a passion. And that’s not good.
You may still end up ‘publishing to appease’ every so often, but don’t let it be your life. Indeed, as @warrenstaples commented, many applied disciplines (such as his: Management) offer research that appears across a broad range of FoR codes, and that is the way it should be, but it’s not a neat package for slicing into FoR categories.
The best way to ‘game’ a system is to learn the rules, and find ways to make what you already do matter.
If you want to know a bit more about the Australian research quality game, ERA, and FoR Codes, RW has previously addressed these topics:
The Research Whisperer 
Two comments:
1) It is crazy for researchers to try to publish in an international journal when their findings are relevant to a local professional audience – a journal read by the latter would be much more relevant and their work will have greater ‘true’ impact (and, as Tseen notes, will be accepted much more rapidly). I applaud the idea of thinking through how you need to build a career, rather than just chasing the immediate solution. This applies to thinking through what kind of grants you will go for as well – building a profile rather than chasing whatever will provide money at the time. Unfortunately, however, expediency sometimes has to win out (especially re funding), in which case, the trick is to shape what you do the best way you can to gain some longer term benefit.
2) I publish methodology articles for which there are no sensible FoR codes. I suspect Tseen has a similar problem publishing about researcher development (which sort of but not really fits into higher education).
LikeLike
Thanks, Pat.
1) Yes, applies very much to grants as well – sometimes more so because the time-commitment to the grant application, doing the project, and producing outcomes (publications) can take many years. With the grants landscape, it can be easy to spot what the next big thing might be (currently in Oz, it’s ‘impact’), but finding ways to best present yourself within new frames of valuation can be tricky.
2) You’re right about researcher development being ‘sort of but not really’ in HE – doing some lit.review into a new area and trying to work out whether the terms I’m using are wrong, or whether there’s just not a lot on that area from HE perspectives…
If you ever feel the urge to write a blogpost on these issues, drop us a line. We’d be particularly interested in your take on research performance measures, given you’ve actively researched in this area.
LikeLike
Hi Pat,
I would only say in response to point 1 that what is relevant for the local audience can also be relevant for an international audience, and many within the local professional audience also read international journals. Choosing to publish locally will tend to limit the international audience, but publishing internationally will not greatly limit the local audience. If research is framed in terms of its theoretical contributions, I think there can be greater citation and “true” impact from publishing domestic results for international audiences.
LikeLike
I take your point Peter. For me, I guess it depends on the orientation of the paper – whether more theoretical or more practical. Some years ago, when researching who were awarded ARC grants, we found that those who had a record of industry/philanthropic etc funding had much lower success rates on ARC discovery (or large, as they were then known – about 3% compared to 20%) than those with a background of other discovery grants. As a research developer, I would therefore always encourage those who had such funding to draw out some of the results from their reports, and to put them into a stronger theoretical framework for refereed journal publication in order to build their academic profile. ECRs in applied disciplines in particular often need to take a route of lower status funding and journals in order to start getting runs on the board, but there is usually something within what they are doing that can be built into a more basic/academic/theoretical output – they might just need some guidance to see it and capitalise on it.
LikeLike
Hi Pat,
It would be interesting to hear more about the relative success rates and characteristics of those who are successful. The ARC would have very good quantitative data on this, but I have not been able to get them to release it to me, it is their policy not to, which leaves research to be conducted on one’s own data which is inherently less reliable and falling back on doing yet another survey with low response rates.
While I agree that ECRs may benefit from starting in lower status journals, people often don’t think they are ready to submit things to a particular high status journal due to a belief that the research is not yet good enough (e.g. perfectionism, or comparing one’s research to the best research published in a journal, rather than the minimum standard).
LikeLike
My data isn’t current, but I suspect the pattern hasn’t changed greatly. It comes from a study of early career researchers carried out for the ARC in 1995. The report is available on my website (www.researchsupport.com.au) and the analysis of who wins ARC grants was published as an article: Bazeley, P. (1998). Peer review and panel decisions in the assessment of Australian Research Council project grant applications: what counts in a highly competitive context? Higher Education, 35, 435-452, as was the final definition of ‘early career’ and its justification: Bazeley, P. (2003). Defining ‘early career’ in research. Higher Education, 45(3), 257-280.
LikeLike
I still “walk in the light” (I think), but I’d be naive to think that ERA did not have some influence, though more in terms of “historic” ranking (A*, A etc. in ERA) than the assigned FOR. For me as an early career researcher, the ERA journals listed under the education and related FORs are a guide for what journals are out there in my field, beyond the ones I already know. If they are not on the list, they don’t exist. If they have historically received A*, A or B, I will look closer at them for potentially relevant journals. But I would have zero hesitation to publish in a FOR out of my own faculty FOR, particularly if it were a highly ranked and relevant journal.
Even though there may be some implications for my faculty, I have never heard anyone suggest a journal based on FOR (though, unsurprisingly, academics are well aware of rankings). Maybe it is an outcome of being at a Go8 where specialisation and concentration of funding within certain FORs is less of an issue.
I think it is very important from a strategic standpoint to publish across a range of FoRs, rather than only within one’s own FOR. Research academics no longer have job security and cannot rely on getting funding for research within their narrow specialisations. By publishing across a range of FORs, it shows that one has research competence that is recognized through peer-review by academics in multiple disicplines. If someone works in sociology and publishes in a range of journals containing FOR codes for sociology, education and organisational behaviour, if funding or a job comes up in education or business faculties, they would have a much better chance of being successful. The faculty offering the job may have some concerns (e.g. “we are looking for someone who will publish in international journals in business (/education)”), but the applicant has a lot stronger chance if they have publications within the given FOR already, even if only a few or in journals with multiple FORs.
There may also be a “reverse snobbery” benefit from publishing more widely. Academics understand the rubbish which gets published in their FOR’s journals, but have less understanding in highly codified knowledge systems in parallel fields (e.g. economics, psychology). Publishing in a top-ranked journal in another discipline may then command even more respect than publishing within one’s own (however, I have no evidence of this to back up my gut feelings on this).
More generally beyond the FOR debate, there is a lot of negativity about ERA, particularly the journal rankings, but I think it is better to have than not have quantitative measures of research output and quasi-quality measures based on impact (citation) and esteem (journal ranking). I know that these are controversial views, but I don’t believe all journals are equal in the quality of the research they publish and I think non-publishing academics should have to justify themselves against their more productive colleagues, and research which is uncited (or only self-cited) should be treated with greater skepticism. Decisions shouldn’t be made entirely on these measures, but they are signals for greater attention.
LikeLike
I’m in vehement agreement with you on most of what your comment, and particularly about the diverse publication across FoRs angle. I have heard of several instances where disciplines are reconsolidating along FoR lines in ways such as: senior staff providing listings of key journals as desired publication outlets (to concentrate output in journals that are in cut’n dried alignment with School/Faculty FoR codes), or specific titles are targeted to boost profile of School, or (clearly) FoR-relevant research is being prioritised. You may be right about the fact that Go8s, which usually have a broader spread of 4s/5s don’t need to be as hardline/strategic about designating FoR codes for maximum effect.
I also agree re the ‘reverse snobbery’ that you mention above. Being seen to have relevance beyond one’s disciplinary home has cachet and – in intellectual terms – is often the root of great scholarship.
LikeLike
Forgot to say: I don’t believe all journals are equal in quality or scope, and do believe that productive academics are an inherently good thing, but I don’t know if the heavy reliance on the metrics we have is necessarily the best way. That’s why I’m particularly interested in alt-metrics ideas, because I think some of this material tries to capture in a more sophisticated way the context of an academic’s work and influence.
LikeLike
[…] are issues, however, with this type of publication strategy. Tseen Khoo, one half of The Research Whisperer blog team, discussed a few of these issues in a post back in 2014. She concluded the post with the […]
LikeLike