fetch API and filesystem: URLs (a Manifest v3 migration blocker)

235 views
Skip to first unread message

Nigel Tao

unread,
Jul 5, 2024, 3:15:11 AM (13 days ago) Jul 5
to Nidhi Jaju, Adam Rice, blink-dev, Chromium Extensions
Hi third_party/blink/renderer/core/fetch/OWNERS (nidhijaju, ricea),
CC'ing blink-dev and chromium-extensions. Feel free to CC other people
or lists if they're relevant.

How hard is it to add filesystem: URL suport to the fetch API? I have
worked with chromium code for some years now, but haven't really
touched blink code generally (and fetch code specifically). For
example, fetch is "a blink thing" and I don't know how problematic
it'll be that Chromium's storage/browser/file_system code lives
outside of third_party/blink.

If it's hard, are there any workarounds to get my XMLHttpRequest-using
extension on to Manifest v3? AFAICT this is a hard blocker for me to
get off Manifest v2.

---

Background:

I work on ChromiumOS' "file manager" app, which uses an "image_loader"
helper extension. This has been MV2 (Manifest v2) for some years now,
and works great.
https://source.chromium.org/chromium/chromium/src/+/main:ui/file_manager/image_loader/manifest.json;l=8;drc=c2aae80e18cbdadde0a3565fa8d5635f2ad378ae

I'm trying to migrate it to MV3 (Manifest v3). This apprently requires
migrating from XHR (XMLHttpRequest) to fetch.
https://developer.chrome.com/docs/extensions/develop/migrate/to-service-workers#replace-xmlhttprequest

https://crrev.com/c/5679823 attempts to do just that (assuming that
fetch should work with either MV2 or MV3). But it doesn't.

ChromiumOS' file manager works with real (kernel-visible) files (e.g.
your local downloads) but also virtual files (e.g. cloud storage like
Google Drive or Microsoft OneDrive, or "connect your phone to your
Chromebook via USB cable and see your files over the MTP, uh,
protocol"). While "filesystem:etc" URLs might have been created 10+
years ago as a web-facing API, the storage::FileSystemURL class
currently powers the ChromiumOS file manager too.
https://source.chromium.org/chromium/chromium/src/+/main:storage/browser/file_system/file_system_url.h;l=83-84;drc=71bd8f1596b301a842247d3488f901d7ae3dfad2

filesystem: URLs work with XHR. They do not work with fetch.

When manually testing the https://crrev.com/c/5679823 change, I get an
error logged:

"Fetch API cannot load filesystem:chrome-extension://etc/etc. URL
scheme "filesystem" is not supported."

This comes from here:
https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/core/fetch/fetch_manager.cc;l=926-928;drc=6c848b551a600625f8b361d1eaad4f8e94d1e40a

It's triggered by passing "RendererCorsIssueCode::kCorsDisabledScheme"
here (although I don't think CORS is actually involved; the error code
is a placeholder):
https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/core/fetch/fetch_manager.cc;l=907-910;drc=6c848b551a600625f8b361d1eaad4f8e94d1e40a

In fact, right next to that trigger is "// FIXME: implement other
protocols". The FIXME dates from 2014:
https://codereview.chromium.org/399543002/diff/370001/Source/modules/serviceworkers/FetchManager.cpp

I'm guessing that, if it hasn't been fixed in 10 years, it's probably
not going to get fixed.

If it's not going to get fixed, is there a workaround?

Nigel Tao

unread,
Jul 5, 2024, 9:05:14 AM (13 days ago) Jul 5
to Nidhi Jaju, Adam Rice, blink-dev, Chromium Extensions
Sending again (this time hopefully to chromium-extensions@).

woxxom

unread,
Jul 5, 2024, 1:18:18 PM (13 days ago) Jul 5
to Chromium Extensions, Nigel Tao, Nidhi Jaju, Adam Rice, blink-dev
You can still use XMLHttpRequest inside the offscreen document.

Oliver Dunk

unread,
Jul 6, 2024, 2:04:54 PM (12 days ago) Jul 6
to Nigel Tao, Nidhi Jaju, Adam Rice, blink-dev, Chromium Extensions
Hi Nigel,

This chromium-extensions list is a public one mostly used for announcements and questions from external developers. Unfortunately, it's not the best place to ask implementation questions. I can definitely sync with you offline and share some additional contacts.

Is there a possibility of using the file:// scheme here? I'm not familiar enough with ChromeOS and the filesystem:// scheme to know if that would work, but that scheme is supported in `fetch` and what we would expect to be used most often. For testing, note that you need to request the `file://*` host permission and enable the "Allow access to file URLs" toggle on the extension's details page.

Thanks,
Oliver Dunk | DevRel, Chrome Extensions | https://developer.chrome.com/ | London, GB


--
You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extens...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/CAEdON6YmUi4sSmNgdS1moO0yXMEb0sfO16Eukjtr3Jm5u0aD5w%40mail.gmail.com.

Nigel Tao

unread,
Jul 7, 2024, 11:38:29 PM (10 days ago) Jul 7
to Oliver Dunk, Nidhi Jaju, Adam Rice, blink-dev, Chromium Extensions
On Sat, Jul 6, 2024 at 1:20 AM Oliver Dunk <olive...@google.com> wrote:
> This chromium-extensions list is a public one mostly used for announcements and questions from external developers. Unfortunately, it's not the best place to ask implementation questions.

Thanks. I'll take chromium-extensions off the CC list and follow up
with your additional contacts.

Nigel Tao

unread,
Jul 7, 2024, 11:46:57 PM (10 days ago) Jul 7
to Oliver Dunk, Nidhi Jaju, Adam Rice, blink-dev
On Sat, Jul 6, 2024 at 1:20 AM Oliver Dunk <olive...@google.com> wrote:
> Is there a possibility of using the file:// scheme here? I'm not familiar enough with ChromeOS and the filesystem:// scheme to know if that would work, but that scheme is supported in `fetch` and what we would expect to be used most often.

Despite sounding similar, the file:// and filesystem:// schemes aren't
completely interchangeable. Furthermore, our manifest.json's
content_security_policy does not list file: (but it does list
filesystem:) and, IIUC, opening up file: would let the extension e.g
read the entire user profile directory.


> For testing, note that you need to request the `file://*` host permission and enable the "Allow access to file URLs" toggle on the extension's details page.

The ChromeOS Image Loader extension is 'built into ChromeOS' and
doesn't actually show in the chrome://extensions page. It's a private
implementation detail of the OS UI (the file manager). Even if it did
show, FWIW, on my other extensions, I don't see a "Allow access to
file URLs" toggle on the details pages.

Thanks anyway for the suggestions.

Nigel Tao

unread,
Jul 7, 2024, 11:55:49 PM (10 days ago) Jul 7
to woxxom, Nidhi Jaju, Adam Rice, blink-dev
On Sat, Jul 6, 2024 at 3:18 AM woxxom <wox...@gmail.com> wrote:
> You can still use XMLHttpRequest inside the offscreen document.

Huh, https://developer.chrome.com/docs/extensions/develop/migrate/to-service-workers#replace-xmlhttprequest
made it sound like XHR wouldn't work with MV3 at all:
"XMLHttpRequest() can't be called from a service worker, extension or
otherwise". Perhaps that documentation was written before the
chrome.offscreen API became available (in Chrome 109)??

I will give this a try, although IIUC chrome.offscreen isn't available
to MV2 (unlike fetch, which is available for both MV2 and MV3), so it
looks like I'm going to have to cook up one big "migrate to MV3"
commit instead of multiple, smaller commits.

Adam Rice

unread,
Jul 8, 2024, 3:05:42 AM (10 days ago) Jul 8
to Nigel Tao, woxxom, Nidhi Jaju, blink-dev

Nigel Tao

unread,
Jul 8, 2024, 9:14:02 PM (9 days ago) Jul 8
to Adam Rice, woxxom, Nidhi Jaju, blink-dev
On Mon, Jul 8, 2024 at 5:05 PM Adam Rice <ri...@chromium.org> wrote:
> It's possible to enable schemes for Fetch using code similar to https://source.chromium.org/chromium/chromium/src/+/main:extensions/renderer/dispatcher.cc?q=%22WebSecurityPolicy::RegisterURLSchemeAsSupportingFetchAPI(extension_scheme);%22&ss=chromium%2Fchromium%2Fsrc, however it would require extensive security review.

Does RegisterURLSchemeAsSupportingFetchAPI affect just one particular
extension or the entire process? If the process, is it the browser
process, renderer process or both?

Adam Rice

unread,
Jul 10, 2024, 1:19:56 AM (8 days ago) Jul 10
to Nigel Tao, woxxom, Nidhi Jaju, blink-dev
Interesting point. It affects the whole render process.

Marijn Kruisselbrink

unread,
Jul 15, 2024, 12:43:13 PM (3 days ago) Jul 15
to Adam Rice, Nigel Tao, woxxom, Nidhi Jaju, blink-dev
Another alternative to fetching (or XHR-ing) a filesystem: URL could be to pass the URL to webkitResolveLocalFileSystemURL (although I believe that method isn't currently available in service workers either), and then call file() on the resultint FileEntry to get a Blob object you can read...

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAC_ixdyXfuLhVMjUVyCmj7u9RT9JnuQv%2BLrzuhS0%3Dr20_ccqEw%40mail.gmail.com.
Reply all
Reply to author
Forward
0 new messages