Skip to main content

The first taste of Rust - A simple you tube downloader

Recently, I just learnt Rust and using it to write a simple youtube downloader with reference from node-ytdl. In this blog, I would like to share the code and how did I make it. You can find the full source code here.

Install development environment

I am using Windows 10 and scoop package manager. Therefore, I use the following commands.
  • Run scoop install rustup-msvc to install rustup.
  • Run setx "%path%;%USERPROFILE%\\scoop\\persist\\rustup\\.cargo\\bin" to add rustup to the path.
  • Restart termial (git-bash in my case) and check the installation with rustup --version; rustc --version; cargo --version
  • Export custom RUST_HOME: export RUSTUP_HOME=$HOME/scoop/persist/rustup/.cargo/bin/rustup
  • Install a toolchain for rustup: rustup toolchain install stable-x86_64-pc-windows-msvc

Setup project

Run cargo new simple_rust_youtube_downloader --bin && cd simple_rust_youtube_downloader to create and navigate to the project.

Add these dependencies to Cargo.toml file

...
[dependencies]
reqwest = { version = "0.11.2", features = ["blocking", "json"] }
url = { version = "2.2.0" }
serde_urlencoded =  { version = "0.7.0" }
json = { version = "0.12.4" }
  • reqwest to make http requests.
  • url and serde_urlencoded to interact with urls.
  • json is not the most popular json library but I found it very easy to use.

Code and comments

use std::{collections::HashMap, fs::File, io::prelude::Write, process::Command, time::Duration};
use url::Url;

fn main() {
	
    //Set the time out long enough to download a video.
    let client = reqwest::blocking::Client::builder()
        .timeout(Duration::from_secs(10 * 60))
        .build()
        .unwrap();
    
    //Get first argument from command line as input either a video link or a video id.
    let mut input = std::env::args().nth(1).expect("no vid id given");
    let mut video_id = "".to_owned();
    
    //Check if the the input contains http/https => link else => id.
    if (&input).starts_with("https://") || (&input).starts_with("http://") {
        match Url::parse(&input) {
            Ok(parsed_url) => {
                let hash_query: HashMap<_ _=""> = parsed_url.query_pairs().into_owned().collect();
                video_id = hash_query.get("v").unwrap().as_str().to_owned();
            }
            Err(e) => eprintln!("{:?}", e),
        }
    } else {
        video_id = input;
    }

    let info_url = format!("https://youtube.com/get_video_info?video_id={}&html5=1", video_id);

    match client.get(&info_url).send().unwrap().text() {
        Ok(v) => {
            let resp_map = serde_urlencoded::from_str::>(&v).unwrap()
                as HashMap;
            let json_player_resp =
                json::parse(&resp_map.get::(&"player_response").unwrap() as &str).unwrap();
            let streaming_data = &json_player_resp["streamingData"]["formats"];
            let video_details = &json_player_resp["videoDetails"];
            println!("video_details: {}", &video_details["title"].to_string());
            let file_name = format!("{}.mp4", &video_details["title"].to_string());
            let normalized_file_name: String = format!(
                "{}",
                file_name
                    .chars()
                    .map(|x| match x {
                        ':' => '-',
                        '"' => ' ',
                        '/' => ' ',
                        '\\' => ' ',
                        '*' => ' ',
                        '?' => ' ',
                        '<' => ' ',
                        '>' => ' ',
                        '|' => ' ',
                        _ => x,
                    })
                    .collect::()
            );
            println!("normalized_file_name: {}", &normalized_file_name);
            if std::path::Path::new(&normalized_file_name).exists() {
                println!("file exist");
                return;
            }
            let mut file = File::create(&normalized_file_name).unwrap();
            match (&streaming_data)[streaming_data.len() - 1]["url"].as_str() {
                Some(url) => {
                    println!("streaming_data: {:?}", &url);
                    let downloaded_video = client.get(url).send().unwrap().bytes().unwrap();
                    file.write_all(&downloaded_video).unwrap();
                }
                None => {
                    download_ciphered_video(&normalized_file_name, &video_id);
                }
            }
        }
        Err(e) => println!("error parsing header: {:?}", e),
    }
}


//Function to use node package ytdl to download ciphered video link.
fn download_ciphered_video(normalize_file_name: &str, video_id: &str) {
    Command::new("node")
        .arg("node_modules/ytdl/bin/ytdl.js")
        .arg(video_id)
        .arg("-o")
        .arg(normalize_file_name)
        .arg("-q")
        .arg("22")
        .output()
        .expect("failed to execute process");
}


The above code will: 

  1. Receive input as video link or video id. 
  2. Fetch the information from info_url "https://youtube.com/get_video_info?video_id=<video_id>"
  3. Choose to download .mp4 format to a file with file name being extracted from the title of the video. If the file is already exist then the program will exit without doing anything.
  4. This project will only handle the easy path to download youtube video. For the more complicated case, I am using node-ytdl. Therefore, there is a need to run npm install ytdl to the folder where the executable program built.
  5. Run test project npm install ytdl && cargo run -- https://www.youtube.com/watch?v=EToeZxIPdKg

This is my "hello world" project in rust. I will need to learn more about idiomatic rust. So far, my feeling working with rust is that it is strict and easy. It is easy because the compiler print very details error messages so that I can search and fix those errors. The downside, in my opinion, is the very slow compile duration. I think rust is fun and worth learning more.


Reference:
node youtube download https://github.com/fent/node-ytdl

- ninjahoahong

Comments

Popular posts from this blog

My books review: "The retrospective handbook"

Overview: The retrospective handbook is a useful collection of tips for preparing and running retrospective for agile teams. The central point of facilitating a retrospective is the preparation. All the details - even the smallest details such as how much paper is available or what the colours of the using pens - play an important role in running a successful retrospective. In such preparation, the most important preparation is the relationship. It requires team members to have high trust with each other to address any problems in great details and find the most suitable solution for those problems. My impression: The tips for running remote retrospective would be really useful for anyone especially startups in my opinion. All the startups I have worked for so far have had some foreign teams either external or internal to save operating money. In addition, there are a lot of remote work to save office renting space and almost all of those startups followed the agile method.

Prepare Lubuntu 20.04 for software development

Prepare Lubuntu 20.04 for software development ninjahoahong After using windows , macosx , and serveral linux ditributions. For me, Lubuntu is the lightest and easiest to set up and run so far. If you search there will be two domains provide lubuntu which are lubuntu.me and lubuntu.net . You should use lubuntu.me which provide the most updated version of lubuntu . In this blog, I will focus on the additional packages after installation. Create Lubuntu 20.04 bootable usb Download Lubuntu 20.04 iso file. Create bootable usb using balena etcher or unetbootin . Boot to the usb and install Lubuntu . This is a starting screen using lxqt . Additional packages There are packages for apt and snap . I prefer apt since the app installed by snap command will take long time in the first launch. Web browser: brave I usually inst