The add-ons contribution program has a shiny new home, and also a shiny new recognition page! This is a permanent home for reporting your contributions to the community, and serves as the official nomination list for new Friend of Add-ons inductees. Contributors on this list are also recognized at the weekly Monday Project Meetings.
Please remember to note down your contributions, and encourage others to do the same!
Reviewer: erosman
Reviews: 1357
I noticed he was killing the queue, but this is on a whole other level. That is incredible, I had no idea, he’s been cleaning the queue by about 45 reviews per day.
Yes, erosman has been on a tear for a while. He’s currently #1 on the reviewer leaderboard with thousands of reviews performed!
Too bad my add-on has fallen outside his grace, now I am still waiting almost a month for a preliminary review instead of the previous 24h/2days when he was reviewing my updates. Such is the way of AMO.
I almost quoted Eddie Vedder’s Rise song…
Apparently your add-on is flagged for admin review, and that queue is running a bit long…sorry for the wait.
No, I am sorry. I wasn’t trying to make you go through all the trouble of checking the situation. The senior reviewer already discussed the situation with me when it happened, he explained very clearly (and multiple times, especially when I was being very obtuse) that the new code I was trying to upload posed security risks which weren’t allowed in AMO (converting unsanitized remote data into code), a bit more here, a bit more there and it required the review to be handled by an admin.
Erosman also told me that he couldn’t do anything while it was still flagged for admin review since he isn’t an admin reviewer, so I just have to keep waiting. Thus comes the “falling outside his grace”.
The only worry about this is that I fear the time it will take for the preliminary admin review to take place might be about the same as the waiting time for a full review (both preliminary and full review queues are handled by admins only), so I might have to wait the same time as if I was waiting for a full review, but I will only get (at best) a preliminary review.
Despite all this, it would be nice to be informed of the flagging on the dashboard and instead of showing the review queue as if I was never flagged, show the flagged review queue instead. One thing is thinking that I am 97 of 156, another completely different is 3 of 12 (flagged for admin review queue, for example)
It’s somewhat unpredictable at present, but you are right that it could take the same amount of waiting time if you’re waiting for prelim or full if your add-on is flagged for admin review. This is because we only have two active admin reviewers at the moment, and they have very limited time. We’re in the process of growing that group so we can fix this problem, but it will take probably a few weeks or more before it happens.
As for the suggestion of moving admin-flagged add-ons to a different queue, that does make sense, though it might also confuse other developers, and the whole review system is pretty complex as it is.
With ~210 add-ons pending full review and more pending flagged preliminary review, just 2 reviewers doing all the work whenever it is feasible for them is quite problematic for both sides.
And without any way of knowing, the flagged add-ons don’t even have any queue criteria; will they be reviewed after all 210 full reviews are done, will they be reviewed after 10 full reviews are done, after each one, etc.? There’s no way of knowing that a flagged add-on will even be reviewed at all, the present picture can even convey the very likely possibility of never being reviewed at all.
When I asked for someone to take a look at my add-on update pending review, no reviewer replied in IRC, same happened when I used the email. Absolutely nothing in return.
And the flagged queue would be everything but confusing, if anything currently it is confusing because the developer has no clue wether he might have his add-on reviewed anytime soon or not. And to make matters worse, developers in this position observe their position in the queue increasing as if someone is bumping ahead or bumping it behind.
The problems that demand an admin review are prefectly understandable, but to be permanently stuck in that position should not happen. The preliminary reviewers should be the ones that set that flag and it should reset once a new version is uploaded because otherwise the preliminary reviewers are completely unable to do anything, especially if the new version has corrected or fixed the problems that led the add-on to be flagged in the first place. Let the preliminary reviewers make their work and leave the admin reviewers (which are only 2) make theirs (which is already plenty). Because, very honestly, the current system that AMO has is absolutely crippled, just 2 admin reviewers for over 200 add-ons? And the preliminary review has only been alright lately because there’s one reviewer blazing through 40+ add-ons per day, otherwise it would also be as choked as the full review queue.
Don’t make the admin flag permanent, reset the flag if the developer uploads a new version. Otherwise, if the developer corrected the problem that led to the admin flag in the first place, the reviewers in the preliminary section cannot do anything, the 2 review admins’ work load is increased without any necessity and the developer grows frustrated of waiting forever to have his add-on reviewed, and in some cases it can even lead to him giving up on AMO or worse, developing for Firefox completely.
And to make matters worse, the prelinimary review queue position is reset everytime a new version is uploaded. This is a problem that has been going unsolved for over 3 years now
https://bugzilla.mozilla.org/show_bug.cgi?id=717495
and it forces developers to not upload new versions if they are about to be reviewed, and only after they upload a new version which has to be reviewed again. This problem doubles the work when he could have just uploaded the new version and not have its queue position reset.
In conclusion;
I’m sure this is a discussion for another topic. 
But…
What if problem wasn’t corrected, the admin flag is removed and the original problem slips through by accident? As an end-user, I’d rather see this err on the safe side and keep the admin flag so that any issue raised with the code/extension is followed through to the end.
IMHO that won’t give you much more information than you currently have, other than letting you know that your add-on is admin flagged, which might be confusing for those that don’t know what it means (“what is this admin review thing? at least the queue is smaller now, maybe it’ll take less time” - which it obviously won’t). I don’t see it speeding up the review process any bit either, as I believe admins almost always pick from the bottom of the queues already.
BTW one such queue wouldn’t work. It would have to be three at least: “admin prelim”, “admin full” and “admin updated”. I’m not sure this is a good idea either because it might lead to losing the relative queue position the extension was in originally. Plus, as with everything, the more queues the harder it’ll be to manage them.
If we go with the “what if’s” we will never see the end of it, fearmongering is never a good argument foundation, but to answer your concern just know that even the admin review is not better at preventing what you described, a very recent example: https://www.reddit.com/r/firefox/comments/3pwcey/firefox_extension_download_manager_s3_asks_for/cwajfls?context=3
Furthermore, the reviewers are in control of flagging the add-on, they know when something needs to be reviewed by an admin or not. If they don’t know then they weren’t well trained and need to be instructed further.
which might be confusing for those that don’t know what it means
“Flagged for admin review” -> Hover the question mark icon, shows tooltip “This review requires a more experienced reviewer”. Queue shows how many are in the flagged queue and what position you are in, you have a general idea of how far away you might be from being reviewed. There’s no confusion but the confusion you are making.
It would have to be three at least
There’s no such thing as the ones you described. The admins are already the same reviewers doing full reviews, no one else. You only need one queue for the flagged add-ons and you can only have a flagged add-on that is in the preliminary review queue, nowhere else.
Plus, as with everything, the more queues the harder it’ll be to manage them.
The queues already exist, they would only be more clear and transparent for the developers.
It is “better” (although that’s definitely not the best word for it), otherwise there would be no point in even having such a distinction. But of course it is not infallible either.
All three main queues - preliminary, full review and updates - are handled by both admin and non-admin reviewers. The admin-flag can be set on add-ons on all three queues, and it does not equal to full review; i.e. an add-on in the prelim queue that’s been admin-flagged will be subject to the same review criteria as those non-admin-flagged in the same queue.
It is “better”
Have yet to see any proof of this. I showed it wasn’t with a recent example, would like to see any reasonable example that supports your “better”, otherwise it is just your opinion with no basis.
The admin-flag can be set on add-ons on all three queues
You wouldn’t need a distinction for flagged add-ons, you would only need to add the information that the add-on is currently pending admin review because it has been flagged and show the flagged add-on queue. I don’t know how much more clear I can be about this.
No matter which queue you are, your add-on is flagged for admin review. You are still in your previous review type -be it full, update or preliminary-, this doesn’t change. It just shows the flag information in your dashboard and the flagged add-on queue information as well. You are now aware of your situation and you can observe the queue progress, have a rough estimation of when your add-on will be reviewed, schedule anything accordingly if needed.
Have you corrected the problem that required an admin review? Upload new version, any reviewer checks it and if it is acceptable it is approved, if it is not and requires again an admin review it gets flagged again.