[Doc] Retrieving configuration from external source

Hi!

I’ve been looking around for a while, but I don’t seem to find any example or documentation to do what I would like to be able to… Any help would be appreciated.

Basically, I’m writing an extension that will add a button on some pages. The core is working fine :slight_smile:. However, what I would like to do, is to keep my configuration on my site and have the extension retrieve it on startup, or on the tab loading. I’ve been trying a number of things, including a background script, but I don’t seem to be able to get things to work as I would like…

In my current content script, I have basically this:

var cfg = null;
fetchConfig();
console.log('DONE');

function fetchConfig() {
	let gettingItem = browser.storage.local.get("config");
	gettingItem.then(onGot, onError);
}

function onGot(item) {
	console.log('in onGot');
	console.log(JSON.stringify(item));
  if(JSON.stringify(item) != "{}"){
	  // We have something in the local storage: load cfg
	  console.log('we are loading cfg');
	cfg = item;
  }else{
	  if(cfg == null){
		console.log('onGot - cfg is null');
		let req = new Request('https://example.com/getconfig.php', {
			method: 'GET',
			headers: { 'Accept': 'application/json' },
			redirect: 'follow',
			referrer: 'client'
		});
		console.log('starting fetch');
		window.fetch(req).then(function(response) {
			console.log('in fetch');
			// .json returns another promise
			return response.json();
		}).then(function(config) {
			console.log('Setting storage');
			browser.storage.local.set({config: config});
			cfg = config;
			console.log('REGISTERED!');
		}).catch(error => { console.log(error); });
		console.log('onGot finished');
	} // cfg is null?
  }
  console.log('returning ongot');
}

function onError(error) {
  console.log(`Error (onError): ${error}`);
  cfg = null;
}

The idea is to have a global variable (cfg) on the page initialized with the storage.local key “config”. If this storage key doesn’t exist (the first time the add-on is invoked), then I would download the config from my host and store it in storage.set. This will mean that the next time I load a page, I won’t have to fetch the resource from the server and will just use the item in storage.local.

I’m sure there’s something wrong, as this doesn’t seem to work and would like to understand if I’m doing the right thing, if there are better/recommended ways to use this approach, or if this is completely wrong.

These are the settings in manifest.json:

  "permissions": [
	"<all_urls>",
	"webRequest",
	"storage"
	]

Many thanks in advance!

I’m pretty sure you can’t use fetch in the content script due to CORS security.

However you can just do in the background script. Background script starts with a browser so if you execute the fetch on the top-level, it will download it every time user opens the browser - if that’s too much, you can check if you have the value stored in the storage first.
Make sure to study anatomy of the web-extensions to understand the basics:

If you still want to do it in the content script, using the global fetch (I’m nut sure that’s the same as window.fetch!) should work, since you have the host permission.

Edit: it’s the same, see https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/Content_scripts#XHR_and_Fetch

In generally more specific error description would be useful (so exactly where your code fails and how it fails).

Thank you both for the help: I’ve managed to have the configuration work by loading it in the background script and accessing the storage from the content script.

The issue I had was that the message:

console.log('starting fetch');

never appeared in the console… what you’re saying about CORS and the global fetch makes sense, but I had given the extension “<all_urls>” permission, and I thought that must have been sufficient. In regards to the global/windows fetch, in the documentation Martin has linked I read:

(emphasis mine), so I suspected that window.fetch() should have worked there. I think I also tried with the global fetch but without much success.

Anyway, I’m happy with loading the configuration in the background (that was the original plan after all), but I have another question for which I will post another request later.

Many thanks, all!

Thank you for correcting me with the CORS :slight_smile:.
I remember some issues with fetch in the content script but now when I’m looking at it, I see it was related only to Chrome and only with blob: url scheme.
This is what I wrote in my code few months ago:

// warning: this won't work in Chrome - "Fetch API cannot load blob:https://maketext.io/927a3f05-4d24-41cd-adb3-fe4d8d4b9dc2. URL scheme must be "http" or "https" for CORS request."
const blob = await (await fetch(url)).blob();

(I really hope I’ve reported it :smiley:)

Could I question why you actually want to keep the configuration on your online server and fetch it from there, and not just include some static “default” configuration in your add-on itself?

Sure! My addon is a generic engine to insert a link in pages on different sites to provide a direct access to resources nested in second- or third-level navigation. Essentially, a helper to minimize navigation time when looking at resources. The way this is implemented is “per site”, whereby I specify in a JSON configuration selectors and regex resources to navigate the DOM of the site and to extract the relevant information. Having the configuration offline allows me to easily add target sites and quickly change the configuration in case the target site changes its DOM. I’m aware that I could just release a new version of the add-on when I’ve added a site or changed a configuration, but having a central repository for the configuration simplifies my management (as support) and allows me to temporarily disable sites that may no longer work while I work on new rules.

My idea would be to have a static add-on that won’t require updates but that will keep on working on new sites and after changed sites. In the past, I had written some GreaseMonkey scripts for the separate sites (still available here: https://userscripts-mirror.org/users/535233/scripts) but with the dismissal (???) of GreaseMonkey and the fact that these scripts were all similar, I would like to try to develop a single, maintainable engine to do the same.

Hope this helps!

A little off-topic, but Mozilla works on a userscript API for add-on developers, so all this could be brought back to live. See https://www.ghacks.net/2019/03/27/firefox-support-user-scripts-api/