WebExtension: XMLHttpRequest / fetch() cookies are not sent even with third party cookies allowed

Hi everyone,

I’m currently maintaining a WebExtension for my organization, which is not registered for the moment.
This plugin integrates into a specific web page, and adds content to it, while maintaining a state on an external server, so that XMLHttpRequests are sent to a cross-origin domain.

Configuration: 60.0 (64-bit) on Debian 4.9.65-3+deb9u2 (2018-01-04) x86_64 GNU/Linux

Until Firefox 59 (and still on Google Chrome), everything worked fine: my XHR sent to the server were sent and the cookies saved without any intervention.
However, since 59 and now 60, XHR are sent without any cookies when sent from a content script.
All third party cookies are allowed in my configuration, and the permission is set in the manifest:

{
  "manifest_version": 2,
  "name": "XMLHttpRequest demo",
  "version": "0.0.1",

  "description": "Demo app",

  "permissions": [
    "*://127.0.0.1/*",
    "cookies"
  ],

  "content_scripts": [
    {   
      "matches": ["https://developer.mozilla.org/*"],
      "js": [
        "index.js"
      ],
      "run_at": "document_start"
    }     
  ]     
}

I have read the WebExtension: XMLHttpRequest issues: No cookies or referrer thread and tried it, with no success.
I also tried the Request API

I have set the withCredentials = true for XHR and { credentials: 'include' } for the Request constructor.

Here is the version with the Request API:

;(async function () {
  'use strict'
  const url = 'http://127.0.0.1'

  async function call2 () {
    try {
      const myHeader = new Headers()
      myHeader.append('Content-Type', 'application/json')

      const myRequest = new Request(url, {
        method: 'GET',
        headers: myHeader,
        credentials: 'include',
        cache: 'no-store'
      })

      const response = await fetch(myRequest)
      const json = await response.json()
    } catch (err) {
      console.log(err)
    }
  }

  await call2()
  await call2()
})()

Now the version with the XHR API:

;(async function () {
  'use strict'
  const url = 'http://127.0.0.1'

  function call () {
    return new Promise((resolve, reject) => {
      const req = new XMLHttpRequest()

      req.addEventListener('load', resolve)
      req.addEventListener('error', reject)
      req.addEventListener('abort', reject)

      req.open('GET', url, true)
      req.withCredentials = true

      req.send(null)
    })
  }
  await call()
  await call()
})()

Here, the First request (caught with Burp Suite):

GET / HTTP/1.1
Host: 127.0.0.1
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0
Accept: */*
Accept-Language: fr-FR,en-US;q=0.7,en;q=0.3
Accept-Encoding: gzip, deflate
content-type: application/json
origin: null
DNT: 1
Connection: close
Ghostery-AntiTracking:  
Pragma: no-cache
Cache-Control: no-cache

Now, the First Response (note that the server is a test one, no cookie is leaked here):

HTTP/1.1 200 OK
X-Powered-By: Express
Access-Control-Allow-Origin: *
Content-Type: application/json; charset=utf-8
Content-Length: 16
ETag: W/"10-66f6f7df"
set-cookie: connect.sid=s%3AHuzyxdcG6OFcjWzw_OujAFh-LzCQe1Zo.MZw3BwBW9QuBoul1vTg1k2QfRj1yD7%2FgrJWmupMHRJQ; Path=/
Date: Mon, 14 May 2018 13:49:36 GMT
Connection: close

{"success":true}

The server defines correctly the cookie in the response.
Now, when the second request is sent, no cookie is sent along with it:

GET / HTTP/1.1
Host: 127.0.0.1
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0
Accept: */*
Accept-Language: fr-FR,en-US;q=0.7,en;q=0.3
Accept-Encoding: gzip, deflate
content-type: application/json
origin: null
DNT: 1
Connection: close
Ghostery-AntiTracking:  
Pragma: no-cache
Cache-Control: no-cache

My question is the following: What is the right, documented way to make authenticated cross origin requests with WebExtensions ?
What did I miss from the docs ?

That looks pretty correct to me. The only thing I find curious is the Ghostery-AntiTracking header:

How is a “AntiTracking” header supposed to work? All it can possibly do is increase your fingerprint surface.

More to the point, this shows that you are not using a clean profile.Firefox profiles tend to get messed up in mysterious ways.(there is a reason that Firefox prompts you to reset the profile if you havn’t used it in a while …)

You should really repeat your tests in a clean profile (firefox -p -no-remote).

If that does not help, bug avoidance is on the menu. Try the request from the background page; try it from a page loaded from the target domain; …

Thanks a lot ! With a new profile, everything works fine again.
I now have to understand how to migrate everything from my previous profile without loss, but that is another problem. (EDIT: the refresh functionality helped a lot).

That being said, I don’t understand how a so fundamental functionnality can disapear because of an aging profile.

Regarding the AntiTracking Header, I agree with you, not sure of its purpose !