Skip to main content

The first taste of Rust - A simple you tube downloader

Recently, I just learnt Rust and using it to write a simple youtube downloader with reference from node-ytdl. In this blog, I would like to share the code and how did I make it. You can find the full source code here.

Install development environment

I am using Windows 10 and scoop package manager. Therefore, I use the following commands.
  • Run scoop install rustup-msvc to install rustup.
  • Run setx "%path%;%USERPROFILE%\\scoop\\persist\\rustup\\.cargo\\bin" to add rustup to the path.
  • Restart termial (git-bash in my case) and check the installation with rustup --version; rustc --version; cargo --version
  • Export custom RUST_HOME: export RUSTUP_HOME=$HOME/scoop/persist/rustup/.cargo/bin/rustup
  • Install a toolchain for rustup: rustup toolchain install stable-x86_64-pc-windows-msvc

Setup project

Run cargo new simple_rust_youtube_downloader --bin && cd simple_rust_youtube_downloader to create and navigate to the project.

Add these dependencies to Cargo.toml file

...
[dependencies]
reqwest = { version = "0.11.2", features = ["blocking", "json"] }
url = { version = "2.2.0" }
serde_urlencoded =  { version = "0.7.0" }
json = { version = "0.12.4" }
  • reqwest to make http requests.
  • url and serde_urlencoded to interact with urls.
  • json is not the most popular json library but I found it very easy to use.

Code and comments

use std::{collections::HashMap, fs::File, io::prelude::Write, process::Command, time::Duration};
use url::Url;

fn main() {
	
    //Set the time out long enough to download a video.
    let client = reqwest::blocking::Client::builder()
        .timeout(Duration::from_secs(10 * 60))
        .build()
        .unwrap();
    
    //Get first argument from command line as input either a video link or a video id.
    let mut input = std::env::args().nth(1).expect("no vid id given");
    let mut video_id = "".to_owned();
    
    //Check if the the input contains http/https => link else => id.
    if (&input).starts_with("https://") || (&input).starts_with("http://") {
        match Url::parse(&input) {
            Ok(parsed_url) => {
                let hash_query: HashMap<_ _=""> = parsed_url.query_pairs().into_owned().collect();
                video_id = hash_query.get("v").unwrap().as_str().to_owned();
            }
            Err(e) => eprintln!("{:?}", e),
        }
    } else {
        video_id = input;
    }

    let info_url = format!("https://youtube.com/get_video_info?video_id={}&html5=1", video_id);

    match client.get(&info_url).send().unwrap().text() {
        Ok(v) => {
            let resp_map = serde_urlencoded::from_str::>(&v).unwrap()
                as HashMap;
            let json_player_resp =
                json::parse(&resp_map.get::(&"player_response").unwrap() as &str).unwrap();
            let streaming_data = &json_player_resp["streamingData"]["formats"];
            let video_details = &json_player_resp["videoDetails"];
            println!("video_details: {}", &video_details["title"].to_string());
            let file_name = format!("{}.mp4", &video_details["title"].to_string());
            let normalized_file_name: String = format!(
                "{}",
                file_name
                    .chars()
                    .map(|x| match x {
                        ':' => '-',
                        '"' => ' ',
                        '/' => ' ',
                        '\\' => ' ',
                        '*' => ' ',
                        '?' => ' ',
                        '<' => ' ',
                        '>' => ' ',
                        '|' => ' ',
                        _ => x,
                    })
                    .collect::()
            );
            println!("normalized_file_name: {}", &normalized_file_name);
            if std::path::Path::new(&normalized_file_name).exists() {
                println!("file exist");
                return;
            }
            let mut file = File::create(&normalized_file_name).unwrap();
            match (&streaming_data)[streaming_data.len() - 1]["url"].as_str() {
                Some(url) => {
                    println!("streaming_data: {:?}", &url);
                    let downloaded_video = client.get(url).send().unwrap().bytes().unwrap();
                    file.write_all(&downloaded_video).unwrap();
                }
                None => {
                    download_ciphered_video(&normalized_file_name, &video_id);
                }
            }
        }
        Err(e) => println!("error parsing header: {:?}", e),
    }
}


//Function to use node package ytdl to download ciphered video link.
fn download_ciphered_video(normalize_file_name: &str, video_id: &str) {
    Command::new("node")
        .arg("node_modules/ytdl/bin/ytdl.js")
        .arg(video_id)
        .arg("-o")
        .arg(normalize_file_name)
        .arg("-q")
        .arg("22")
        .output()
        .expect("failed to execute process");
}


The above code will: 

  1. Receive input as video link or video id. 
  2. Fetch the information from info_url "https://youtube.com/get_video_info?video_id=<video_id>"
  3. Choose to download .mp4 format to a file with file name being extracted from the title of the video. If the file is already exist then the program will exit without doing anything.
  4. This project will only handle the easy path to download youtube video. For the more complicated case, I am using node-ytdl. Therefore, there is a need to run npm install ytdl to the folder where the executable program built.
  5. Run test project npm install ytdl && cargo run -- https://www.youtube.com/watch?v=EToeZxIPdKg

This is my "hello world" project in rust. I will need to learn more about idiomatic rust. So far, my feeling working with rust is that it is strict and easy. It is easy because the compiler print very details error messages so that I can search and fix those errors. The downside, in my opinion, is the very slow compile duration. I think rust is fun and worth learning more.


Reference:
node youtube download https://github.com/fent/node-ytdl

- ninjahoahong

Comments

Popular posts from this blog

Prepare Lubuntu 20.04 for software development

Prepare Lubuntu 20.04 for software development ninjahoahong After using windows , macosx , and serveral linux ditributions. For me, Lubuntu is the lightest and easiest to set up and run so far. If you search there will be two domains provide lubuntu which are lubuntu.me and lubuntu.net . You should use lubuntu.me which provide the most updated version of lubuntu . In this blog, I will focus on the additional packages after installation. Create Lubuntu 20.04 bootable usb Download Lubuntu 20.04 iso file. Create bootable usb using balena etcher or unetbootin . Boot to the usb and install Lubuntu . This is a starting screen using lxqt . Additional packages There are packages for apt and snap . I prefer apt since the app installed by snap command will take long time in the first launch. Web browser: brave I usually inst

My books review: Money for the rest of us

Overview: The "Money for the rest of us" book is a list of 10 questions serve as a guideline to invest. It is light book with not many pages and can be read during commute time. The book does not require audiences to have any special knowledge. My impression: The author, Mr. David, is very friendly and patient to answer my questions. The 10 questions are greatly explained by the author, and I think they provide a good starting point. However, I think anyone who wants to invest should build their own checklist and a general guideline for different situations. When a general guideline is in place, the decisions need to respect the guideline in order to make consistent decision. Consistency , in my opinion, is the most important in the long run as it provide information for constant feedback and improvements. Overall, I think this is a good book that helps improving my decision making process. - ninjahoahong