Proposal: Enable declaration files for non-js-extensioned files

[weswigham]

Suggestion

Ref #49970 and confusion therein.

⭐ Suggestion

Today, TS will only load declaration files for .js, .cjs, and .mjs files (and TS+jsx equivalents). There is no actual mechanism for specifying type definitions for a relative import for files with another extension - nonrelative imports can be defined using an ambient module declaration (see the widely used declare module "*.css";), or via import or export map entry, while relative imports have no mechanism available.

In traditional cjs resolution, this wasn't obviously a problem, since if you wrote require("./style.css"), we would fail to load the .css file (it's not a javascript or typescript extension so we wouldn't even try), and fall back to cjs directory and extension resolution - meaning we'd eventually look for "./style.css.js" and accompanying "./style.css.d.ts", which was close enough for most people's purposes. Now, we have a prominent runtime that does not have extension searching (esm in node/some bundlers/browsers), where there's no mechanism to provide types for the specifier, but in some conditions do support importing non-js extension files.

📃 Motivating Example

import {whatever} from "./stylesheet.css";
import {whatever} from "./mod.wasm";
import {whatever} from "./component.html";
import {whatever} from "./db.json";

💻 Use Cases

Mostly bundlers and potentially browsers for css/html imports, but also node for the potential to type relative imports of .wasm/.json/.node via declaration file, or other extensions as allowed via custom loader.

📃 Proposal

Generally speaking, we want to keep a 1:1 mapping of runtime file extension to declaration file extension, this way we can always do a simple side-by-side lookup for the declarations for a file imported, a relatively simple mapping of output declaration path back to input filepath, and also capture any important format information implied by the original extension in the declaration filename (eg, if .wasm imports end up being importable as uninstantiated modules in the future, and other modules aren't). As such, any mechanism needs to be fairly unambiguous.

Given that, I'd have to propose that a

filename.ext

maps to a

filename.d.ext.ts

which is a declaration file (we'd have to update our definition of declarations to be .d.ts, .d.cts, .d.mts, and .d.*.ts).

This does a few things:

1. It allows any ext to have a unique TS equivalent
2. Unlike a filename.d.ts, it doesn't also ambiguously map to a filename.js or filename.otherext
3. Unlike a filename.ext.d.ts, it doesn't already map to a filename.ext.js
4. Unlike a filename.d.ext, it retains a well-known .ts final extension, for relatively painless tooling and editor support

There are some considerations:

• You could already have a filename.d.ext.ts source file today - it would be a TS source file (that emits a filename.d.ext.js and a filename.d.ext.d.ts) and not a declaration file, making this a technically breaking change. Such a filename is so unwieldy as to be unlikely in my view though, so I don't think this break should be too bad. In some ways, this is an upside, since external tooling will already recognize and parse these files as TS without modification, even if they don't see them as declarations yet.
• Unfortunately, unless we specify otherwise, this does imply you should be able to have a filename.d.js.ts, which behaves identically to a filename.d.ts, and we'd have to prioritize one of the two during lookup. Tentatively, I'd say we should just forbid ts and js extension patterns like this, so .d.js.ts, .d.mjs.ts, .d.cjs.ts, .d.jsx.ts, .d.ts.ts, .d.tsx.ts, .d.mts.ts, and .d.cts.ts shouldn't be looked up - the existing short forms take their place.
• In a multi-module-format situation like in modern node, it's also an open question what format these modules should be interpreted as, which at runtime depends on if the loader used injects the module into the cjs require cache or only synthesizes an esm dynamic module. Because of that distinction (and the inability to import an esm-format-only thing from a cjs format thing), it may be important to encode the expected runtime format into the declaration name as well. For that reason, it would be tempting to reuse the mts and cts extensions; unfortunately this introduces some ambiguity, as all of filename.d.ext.ts, filename.d.ext.mts, and filename.d.ext.cts would map to the same original filename.ext, rather than only one canonical one (and looking up the input filepath from an output declaration file is critical for scenarios like project references, which is why unambiguousness is so helpful here). We could just assume these declaration files are formatless, like ambient modules, and provide the same interface to both cjs and esm callers. That might be good enough to get by. Failing that, the only thing I can think of that preserves filename uniqueness would be some kind of in-declaration-file pragma for asserting the format of the containing file, or a compiler option specifying a global mapping of extension to format (though the later doesn't hold up well with redistributable libraries). But all that may be unnecessary, since, with the exception of potentially .json, these should largely be authored by hand, so protection from format misuse is maybe less important than with JS files writ large.
• These mappings should be enabled and looked up in all resolution modes - classic, node, node16, nodenext. It may, however, be appropriate to add an error on non-js non-declaration imports than is only suppressed with a compiler option (eg, allowNonJsImports, in the same vein as resolveJsonModule).
• Alongside this change, we should also be able to enable our long-supported-but-disabled declaration emit for json documents when resolveJsonModule is set, since it would canonicalize filename.d.json.ts as the declaration path for the json (and thus canonicalize loading such a declaration file at higher priority than the original json document in a wildcard include pattern).

Thoughts?

[barak007]

I think it's great to support definition files for other file types.
I very much agree with the one to one mapping, it's the only way forward. But I think the proposed solution is too complex.

A solution that replaces the file extension like x.d.ts for x.js is not ideal because it's a system that cannot support files without extension or requests where the extension is not provided (resolved extension). It's also difficult and costly to identify the real filename or its type. it's really only a convention, it's context dependent, and only if you know to look for it then you'll find it, and you'll have to search for it every time. Any integration that searches files by extension will have to be aware to maybe not process files that contain double (or as proposed, triple) extensions.

On top of that, the proposed solution to add another extension will create many more extensions to look for, (such as, .d.css.ts, .d.wasm.ts), and makes the process of finding if a file is definition file or not, more costly (3 extname calls).

My perhaps slightly radical proposal is to use a completely new dedicated extension like .dts or .tsd, for definition files. I think it will make the process of matching them simple and elegant. All the metadata of the module type (cjs, mjs) is in the original filename, and finding a definition file for a real file is just to add the extension and consider rootDirs. The other way around is just one extname call.
Proposal Example:

/path/to/file     -> /path/to/file.dts
/path/to/file.ext -> /path/to/file.ext.dts

Output:
/path/to/file.ts -> /path/to/file.js
                    /path/to/file.js.map
                    /path/to/file.js.dts
                    /path/to/file.js.dts.map

/path/to/file.mts -> /path/to/file.mjs
                     /path/to/file.mjs.map
                     /path/to/file.mjs.dts
                     /path/to/file.mjs.dts.map

/path/to/file.cts -> /path/to/file.cjs
                     /path/to/file.cjs.map
                     /path/to/file.cjs.dts
                     /path/to/file.cjs.dts.map

[weswigham]

I don't think we'd want to change our existing handling of ts/tsx/mts/cts/js/jsx/mjs/cjs, so they'd be an exception to any open-ended thing we do, regardless of the exact scheme.

Otherwise, I'm not wholly opposed to a new extension like .dts, since introducing .mts and .cts went pretty OK over the last year, but it does break the pretty consistent .d. pattern used in all other declaration file names today (and lacks the free tooling recognition as typescript that a .ts last extension would give). .dts, specifically, is also already used for linux kernel stuff - device trees, I think.

[dummdidumm]

For Svelte we use the current system to create type declaration files next to their Svelte implementation in the case of npm library packaging:

src/Hello.svelte
-->
dist/Hello.svelte
dist/Hello.svelte.d.ts

This works great for us. If I understand correctly, the new proposal would be to have a Hello.d.svelte.ts file (or Hello.svelte.dts in the other proposal). Would the old resolution still exist for backwards-compatibility, or would that be a breaking change from say 4.9 to 5.0?

[idoros]

Hello.svelte.d.ts is already not picked up today in case you have moduleResolution: node16 configured

[DanielRosenwasser]

Technically we now have working go-to-definition for non-JS files. I assume the .d.ext.ts files will now take precedence under this mode. Given the feedback in #15146, should we change the behavior, or serve up both?

[idoros]

@DanielRosenwasser , you mean for the case that both a global declare module '*.ext' and a .d.ext.ts are found?

[weswigham]

@DanielRosenwasser if a .d.ext.ts file is present, it should take priority (especially since it actually contains the type information), and if the author of such a file wants to go to the original file (because the declarations are generated in some way), they can always include a declaration source map that maps back to the non-declaration file in some way.

@idoros A declare module '*.ext' only applies to nonrelative imports, and only if an actual file lookup for the import fails. So the file definitely has priority if it exists in that case.

[tomrav]

@DanielRosenwasser if a .d.ext.ts file is present, it should take priority (especially since it actually contains the type information), and if the author of such a file wants to go to the original file (because the declarations are generated in some way), they can always include a declaration source map that maps back to the non-declaration file in some way.

This is exactly our case with Stylable. From our source *.st.css stylesheets, we generate *.st.css.d.ts files and then create *.st.css.d.ts.map files to map back to the .st.css source. We're definitely not lacking for suffixes, but it works.

[DanielRosenwasser]

we generate *.st.css.d.ts files and then create *.st.css.d.ts.map files to map back to the .st.css source

I don't want to get too off topic, but since the UX discussion is relevant - have you considered using inline source maps in the generated .d.ts file?

[tomrav]

We did consider inlining them, but as far as we could tell back then (and now, when re-testing) it doesn't actually seem to work in VSCode.

Trying to go-to-definition brings me to the .d.ts file and not the source one. I can open an issue for this if you feel like it should be working.

[andrewbranch]

Closed by #51435

[omgitskuei]

This would be really great for better support of MP3 files by TS. Current use case is adding a
<audio></audio> audio player html next to a hard to pronounce name but TS keeps throwing error 2307 because it "Cannot find module or its corresponding type. Various online solutions that don't involve webpack even as recent as 2022 don't work. Within codesandbox.io, the audio player will load the file correctly despite the TS error in the editor, but nothing works for NextJS-TS-React projects. I've talked to NextJS and they said its a TS issue. Your docs mention adding --allowArbitraryExtensions to the compiler but there's no explanation or code to show what you mean by this.
Would it make sense if you add a section to tsconfig.json thats a map<string, string> eg called fileTypes matching file extensions to XYZ?
For the codesandbox.io showing the error but working audio players for both local and external audio files; see here