So far i have seen quite a bit about each section of B2G minus QA. I was working on QA back before the transition of B2G but currently cant find any info on what is going on with QA. Currently i am looking to figure out what is going on with QA and if nothing or very little then start putting B2G QA back together.
Hello @charja13 
Indeed, not a lot of discussion about QA right now. To be honest I don’t know how it was perform before, and I think you’re right, it starts to be a good time to think about QA.
Could you explain to us (or link to resources) about how QA was organized before ? Thanks 
well i can link you some of it but also we pretty much had cycles, every day we would run certain tests, both manual and automated. Pretty much every day we had two builds just for QA reasons. One in the morning so that we could run smoke tests to see if any changes that landed from last night had any major impacts or created any blockers. we would have one in the evening so that if there were any major changes that messed something up in the morning we would have a working build in the evening. We could probably dim it down to one build a day for the sake of sanity tests and smoke tests. now automated test i believe could be run in a simple way, either we can setup a cycle to run certain automated test for each day of the week or set up a script with all of the automated to run every day on the daily QA build. for the most part we had a Automated team and a manual testing team. The manual testers would usually generate the test cases and sometimes the automated team would convert them into automated tests while some test cannot be automated. Used to when a bug was found that has not been encountered before we would generate a test case so that it would be caught if it were to happen again. Currently i already know that the chances of having a decent size QA team are not exactly the most likely, we didnt even have a very large team when employees were still on. So currently i would recommend us putting together a small team as soon as possible, as we start to put b2g back together we will need to have QA team even if it is small as a lot of code will be changing and the possibility of bugs occurring will be a lot higher till we have it back into a stable state.
2 Likes
https://wiki.mozilla.org/B2G/QA is the main QA page, we can still use all of the same processes that were used before but will have to adapt them as things change.
1 Like
@charja13 Thank you for bringing this up, QA is going to be very important as we re-build B2G.
We have no full time Mozilla employees currently working on QA for B2G (except maybe on the 2.6 branch for TV) so it’s really up to people like you take the lead here. And it looks like you just volunteered!
What do you think should be the first step? A manual daily smoke test to look for regressions?
These are the absolute minimum requirements that should be working right now:
- As a user I want to turn on the device and see the homescreen so I can launch websites
- As a user I want to configure WiFi so I can connect to the Internet
- As a user I want to set the correct time on my device so I can use it to tell the time
- As a user I want to navigate to a web page
- As a user I want to search the web
- As a user I want to enter text with the keyboard
- As a user I want to add sites to the homescreen
- As a user I want to make a phone call by dialing a number into the dialer
Do you have a Sony Xperia Z3 compact device you can use for testing?
If you’d like to attend our weekly meetings we can add a section on QA to discuss the current status and next steps.
Let us know what you need.
Ben
1 Like
currently i do not have an aries device, and im guess the chances of me getting one are slim and none sadly. right now im setting up my computer for a flame build environment so daily smoke tests are going to be possible soon. I have gone through and pulled some of the older automated tests off of task cluster so that i can look at them and start building my own, i still have to learn automation currently as i have not work with automation. hopefully in the next month or so i will be able to automate the daily smoke tests. Im going to base them on either b2g desktop or mullet since i dont know how to throw stuff like py tests cases at a device. currently im going to keep using the fxosqa channel if anyone wants to hop over there and give a hand. if not im going to create a separate b2gqa channel. right now the main issue is going to be getting a system in order. Also i would love to attend the meetings, i was attending the weekly QA meetings before they ended as the sunsetting finished up.
Great, so please join us at the meeting tomorrow at 16:00 UTC
Hello,
I’d like to join QA too! I am currently helping test Nightly for Desktop and Android.
I am trying to flash a B2G build in my Flame device without success. I use Windows, so I tried with a virtual Linux machine and the B2G installer but it always fails along the way.
I still have to try starting the system with a Linux DVD or the like but I need a lot of guidance as I am no techie.
Regards,
Gaby
I would greatly appreciate your help!
A while ago I tried yet again with the VM. It failed in the middle of the
blobs extraction (50%) and I couldn’t get it to go so far again. I closed
everything and started again, then I couldn’t even load the installer 
Meeting is a good idea, time may be an issue though!
ill talk to manel, when would be a good time of week for you?
Tuesday or Thursday would work better for me, maybe the other days too,
depending on the time. My time zone is -3 UTC, how about yours?
My timezone is utc -5, if manel is jumping in its gonna be a little more challenging as she is Europe, so we are going to have to do it late in the evening so it’s in the morning g for her
Well, we’ll see… I think afternoon for us is night for her.
okay, i wanted to give and update. Ive done some digging around and some work on getting some automation up and running, sadly all i currently have is bad news, none of the tests setups will work unless we fix marionette as both mochitests and xpcshell depend on marionette to run. so it is looking like the best option is rapidly becoming to fix marionette, the only other option would be to take it as the opportunity to implement a new testing harness. I personally would like to stick with marionette and get it fix since we have documentation for it already and its already built into our code. I have one request to the dev team, when you are rebuilding please build your code with testing in mind. Also @gaby2300 send me a message on irc and ill see what i can do with helping you with getting your device up to date. also if anyone could take a look at marionette and see if you can help me with telling what i need to do to get it back up and running.
1 Like