Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Aria2c, but for real now #220

Draft
wants to merge 69 commits into
base: v3.0
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
0cbc962
first draft of DownloadManager
lukaarma Sep 5, 2020
5334293
first draft of Decrypter
lukaarma Sep 5, 2020
67573fc
installed WebSocket and updated Tmp
lukaarma Sep 5, 2020
685fa27
added debug and best quality flags
lukaarma Sep 5, 2020
0d8b420
changed UserAgent version
lukaarma Sep 5, 2020
6c0e37a
created WebSocket/Aria2c errors
lukaarma Sep 5, 2020
29a6fab
minor formatting changes
lukaarma Sep 5, 2020
d29bd54
very simple test for SIGINT
lukaarma Sep 5, 2020
5350bc3
added debug logging
lukaarma Sep 5, 2020
0c65ff7
added parsing of m3u8 file down to a list of links
lukaarma Sep 5, 2020
903f2bf
updated destreamer to use the new download method
lukaarma Sep 5, 2020
1dff41c
Merge branch 'master' of https://github.com/snobu/destreamer into ari…
lukaarma Sep 7, 2020
6e874f5
exclude debug launch config
lukaarma Sep 7, 2020
9e25870
updated packages
lukaarma Sep 7, 2020
d037b7c
renamed and finished decrypter
lukaarma Sep 9, 2020
8b61f86
added debug logging
lukaarma Sep 9, 2020
aa21e54
added quality option
lukaarma Sep 9, 2020
a185f51
fixed typo and added comment
lukaarma Sep 9, 2020
96f4c90
- removed useless function/properties
lukaarma Sep 9, 2020
ec099e9
- fixed progress bar not updating
lukaarma Sep 9, 2020
f1476ff
done decryption/merging of video/audio/sub traks
lukaarma Sep 9, 2020
a93b328
updating comments
lukaarma Sep 9, 2020
e9dea14
removed bug in quality selection
lukaarma Sep 9, 2020
16a8532
removed useless old code
lukaarma Sep 9, 2020
6f082e1
marked unused code
lukaarma Sep 9, 2020
796753f
change user agent
lukaarma Sep 12, 2020
6c8628e
code cleanup
lukaarma Sep 19, 2020
7cab44a
added debug statement
lukaarma Sep 19, 2020
9453458
installed portfinder
lukaarma Sep 19, 2020
3e472f9
created error for no port aviable
lukaarma Sep 19, 2020
8df5155
implemented port finding
lukaarma Sep 19, 2020
c9c9fef
removed random useless linting test
lukaarma Sep 19, 2020
af4725c
fixed linting errors
lukaarma Sep 19, 2020
38edbad
check for aria existance
lukaarma Sep 20, 2020
482a506
fixed bug on hanging on shutdown
lukaarma Sep 20, 2020
95c7150
improved shutdown sequence
lukaarma Sep 20, 2020
14cfe7c
typo fix
lukaarma Sep 24, 2020
c7e0415
process.exit on uncaught exceptions
lukaarma Sep 24, 2020
502565d
better webSocket initialization
lukaarma Sep 24, 2020
020518e
minor comments and variables name changes
lukaarma Sep 25, 2020
331efd9
fix for *nix platforms
lukaarma Sep 25, 2020
b89e041
workaround for listeners not removed
lukaarma Sep 27, 2020
8848f29
Update README.md
rohit404404 Oct 3, 2020
389be33
Update README.md
rohit404404 Oct 3, 2020
ec24ff9
Merge pull request #242 from rohit404404/patch-1
lukaarma Oct 3, 2020
22968f4
Misc fixes (#244)
lukaarma Oct 8, 2020
15c4203
Update report-trouble.md (#240)
lukaarma Oct 8, 2020
2a3c3eb
fixed buffer maxLenght exceded on aria2c daemon
lukaarma Oct 9, 2020
3249759
Merge branch 'master' of https://github.com/snobu/destreamer into ari…
lukaarma Oct 11, 2020
85f3bea
added access token refresh logic (closes #255)
lukaarma Oct 11, 2020
41206a9
fixed stupid session refreshing typo
lukaarma Oct 11, 2020
1763dc8
changed exec in favour of spawn for aria2c
lukaarma Oct 11, 2020
a179bdc
Update Errors.ts
rpaulucci3 Oct 17, 2020
eacb2b6
Merge pull request #265 from rpaulucci3/patch-1
lukaarma Oct 17, 2020
4a45c68
Fix group download, added 100 video limit
snobu Nov 14, 2020
0f2d902
Fix verbose logging when we get JSON back
snobu Nov 14, 2020
461df7b
Fix for group download IRL this time
snobu Nov 14, 2020
0b4c900
Update README.md
snobu Dec 2, 2020
ddafc50
fixed parsing for group with more than 100 videos
lukaarma Dec 3, 2020
08364ed
new version of error retry, couldn't test it
lukaarma Dec 3, 2020
6d28b3c
updated all packages
lukaarma Dec 3, 2020
c1a0994
add aria2 to prereqs
fulminemizzega Dec 7, 2020
66a7f60
fix quotes in input file example in README.md
fulminemizzega Dec 7, 2020
195d0dd
Added `aria2` prerequisite
pastacolsugo Dec 26, 2020
1e8db6f
Merge pull request #303 from pastacolsugo/patch-1
lukaarma Jan 8, 2021
6d8f3c6
Fix return HTTP 403 reason with or without verbose (#316)
snobu Jan 26, 2021
3fe64d1
Merge branch 'aria2c_forRealNow' into patch-1
lukaarma Feb 9, 2021
cea18bb
Merge pull request #289 from fulminemizzega/patch-1
lukaarma Feb 9, 2021
ffd76ba
Fix a small typo in terminal output (#363)
MahmoudAFarag Apr 7, 2021
File filter

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 19 additions & 1 deletion .github/ISSUE_TEMPLATE/report-trouble.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,24 @@ assignees: ''

---

## PLEASE NEVER PASTE YOUR ACCESS TOKEN INTO A GITHUB ISSUE AS IT MAY CONTAIN PRIVATE INFORMATION
<!--
# BEFORE OPENING A NEW ISSUE CHECK THE EXISTING ONES AND RUN DESTREAMER WITH THE -v/--verbose flag and paste down below the output

# NEVER PASTE YOUR ACCESS TOKEN INTO A GITHUB ISSUE AS IT MAY CONTAIN PRIVATE INFORMATION
When you paste in output from destreamer, locate your access token (it looks like this: `Authorization: Bearer eyJ....<a lot more base64 encoded text>.....`) and redact it.

# Please fill the form below to give us some more info.
-->

OS:

Launch command used:


<details>
<summary>Verbose log</summary>

```
PASTE VERBOSE LOG HERE
```
</details>
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
*.js
*.zip

.vscode\launch.json

.chrome_data
node_modules
videos
Expand Down
20 changes: 0 additions & 20 deletions .vscode/launch.json

This file was deleted.

46 changes: 24 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,9 @@
<img src="https://github.com/snobu/destreamer/workflows/Node%20CI/badge.svg" alt="CI build status" />
</a>

# BREAKING
# destreamer v3.0 (aria2c as download manager)

**destreamer v3.0** is just around the corner. Download speed improvement is astonishing and we have a never before seen photo from the design sessions:<br><br>
![desilva](https://user-images.githubusercontent.com/6472374/93003437-54a7fd00-f547-11ea-8473-e4602993e69d.jpg)

Help us pick a codename for the new release:<br><br>
![codename](https://user-images.githubusercontent.com/6472374/93003896-20ced680-f54b-11ea-8be1-2c14e0bd3751.png)<br><br>
Comment in this thread: https://github.com/snobu/destreamer/issues/223
## This is a pre-release branch so don't expect stability, do expect speed improvements. Tons of it.

![destreamer](assets/logo.png)

Expand Down Expand Up @@ -49,6 +44,8 @@ Hopefully this doesn't break the end user agreement for Microsoft Stream. Since
- **npm**: usually comes with Node.js, type `npm` in your terminal to check for its presence
- [**ffmpeg**][ffmpeg]: a recent version (year 2019 or above), in `$PATH` or in the same directory as this README file (project root).
- [**git**][git]: one or more npm dependencies require git.
- [**aria2**][aria2]: present in your `$PATH`, on Linux you can install via `sudo apt install aria2`.


Destreamer takes a [honeybadger](https://www.youtube.com/watch?v=4r7wHMg5Yjg) approach towards the OS it's running on. We've successfully tested it on Windows, macOS and Linux.

Expand Down Expand Up @@ -108,31 +105,35 @@ Options:
--help Show help [boolean]
--version Show version number [boolean]
--username, -u The username used to log into Microsoft Stream (enabling this will fill in the email field for
you) [string]
--videoUrls, -i List of video urls [array]
you). [string]
--videoUrls, -i List of urls to videos or Microsoft Stream groups. [array]
--inputFile, -f Path to text file containing URLs and optionally outDirs. See the README for more on outDirs.
[string]
--outputDirectory, -o The directory where destreamer will save your downloads. [string] [default: "videos"]
--outputTemplate, -t The template for the title. See the README for more info.
[string] [default: "{title} - {publishDate} {uniqueId}"]
--outputDirectory, -o The directory where destreamer will save your downloads [string] [default: "videos"]
--keepLoginCookies, -k Let Chromium cache identity provider cookies so you can use "Remember me" during login
--keepLoginCookies, -k Let Chromium cache identity provider cookies so you can use "Remember me" during login.
Must be used every subsequent time you launch Destreamer if you want to log in automatically.
[boolean] [default: false]
--noExperiments, -x Do not attempt to render video thumbnails in the console [boolean] [default: false]
--simulate, -s Disable video download and print metadata information to the console[boolean] [default: false]
--verbose, -v Print additional information to the console (use this before opening an issue on GitHub)
--noExperiments, -x Do not attempt to render video thumbnails in the console. [boolean] [default: false]
--simulate, -s Disable video download and print metadata information to the console.
[boolean] [default: false]
--closedCaptions, --cc Check if closed captions are aviable and let the user choose which one to download (will not
ask if only one aviable) [boolean] [default: false]
--noCleanup, --nc Do not delete the downloaded video file when an FFmpeg error occurs [boolean] [default: false]
--verbose, -v Print additional information to the console (use this before opening an issue on GitHub).
[boolean] [default: false]
--closedCaptions, --cc Check if closed captions are available and let the user choose which one to download (will not
ask if only one available). [boolean] [default: false]
--noCleanup, --nc Do not delete the downloaded video file when an FFmpeg error occurs.[boolean] [default: false]
--vcodec Re-encode video track. Specify FFmpeg codec (e.g. libx265) or set to "none" to disable video.
[string] [default: "copy"]
--acodec Re-encode audio track. Specify FFmpeg codec (e.g. libopus) or set to "none" to disable audio.
[string] [default: "copy"]
--format Output container format (mkv, mp4, mov, anything that FFmpeg supports)
--format Output container format (mkv, mp4, mov, anything that FFmpeg supports).
[string] [default: "mkv"]
--skip Skip download if file already exists [boolean] [default: false]
--skip Skip download if file already exists. [boolean] [default: false]
```

- both --videoUrls and --inputFile also accept Microsoft Teams Groups url so if your Organization placed the videos you are interested in a group you can copy the link and Destreamer will download all the videos it can inside it! A group url looks like this https://web.microsoftstream.com/group/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX

- Passing `--username` is optional. It's there to make logging in faster (the username field will be populated automatically on the login form).

- You can use an absolute path for `-o` (output directory), for example `/mnt/videos`.
Expand Down Expand Up @@ -177,9 +178,9 @@ These optional lines must start with white space(s).
Usage -
```
https://web.microsoftstream.com/video/xxxxxxxx-aaaa-xxxx-xxxx-xxxxxxxxxxxx
-dir=videos/lessons/week1
-dir="videos/lessons/week1"
https://web.microsoftstream.com/video/xxxxxxxx-aaaa-xxxx-xxxx-xxxxxxxxxxxx
-dir=videos/lessons/week2"
-dir="videos/lessons/week2"
```

### Title template
Expand Down Expand Up @@ -214,7 +215,7 @@ iTerm2 on a Mac -

![screenshot](assets/screenshot-mac.png)

By default, downloads are saved under `videos/` unless specified by `-o` (output directory).
By default, downloads are saved under project root `Destreamer/videos/` ( Not the system media Videos folder ), unless specified by `-o` (output directory).

## Contributing

Expand All @@ -229,6 +230,7 @@ Please open an [issue](https://github.com/snobu/destreamer/issues) and we'll loo
[xming]: https://sourceforge.net/projects/xming/
[node]: https://nodejs.org/en/download/
[git]: https://git-scm.com/downloads
[aria2]: https://aria2.github.io
[wsl]: https://github.com/snobu/destreamer/issues/90#issuecomment-619377950
[polimi]: https://www.polimi.it
[unipi]: https://www.unipi.it/
Expand Down
Loading