Skip to main content

The first taste of Rust - A simple you tube downloader

Recently, I just learnt Rust and using it to write a simple youtube downloader with reference from node-ytdl. In this blog, I would like to share the code and how did I make it. You can find the full source code here.

Install development environment

I am using Windows 10 and scoop package manager. Therefore, I use the following commands.
  • Run scoop install rustup-msvc to install rustup.
  • Run setx "%path%;%USERPROFILE%\\scoop\\persist\\rustup\\.cargo\\bin" to add rustup to the path.
  • Restart termial (git-bash in my case) and check the installation with rustup --version; rustc --version; cargo --version
  • Export custom RUST_HOME: export RUSTUP_HOME=$HOME/scoop/persist/rustup/.cargo/bin/rustup
  • Install a toolchain for rustup: rustup toolchain install stable-x86_64-pc-windows-msvc

Setup project

Run cargo new simple_rust_youtube_downloader --bin && cd simple_rust_youtube_downloader to create and navigate to the project.

Add these dependencies to Cargo.toml file

...
[dependencies]
reqwest = { version = "0.11.2", features = ["blocking", "json"] }
url = { version = "2.2.0" }
serde_urlencoded =  { version = "0.7.0" }
json = { version = "0.12.4" }
  • reqwest to make http requests.
  • url and serde_urlencoded to interact with urls.
  • json is not the most popular json library but I found it very easy to use.

Code and comments

use std::{collections::HashMap, fs::File, io::prelude::Write, process::Command, time::Duration};
use url::Url;

fn main() {
	
    //Set the time out long enough to download a video.
    let client = reqwest::blocking::Client::builder()
        .timeout(Duration::from_secs(10 * 60))
        .build()
        .unwrap();
    
    //Get first argument from command line as input either a video link or a video id.
    let mut input = std::env::args().nth(1).expect("no vid id given");
    let mut video_id = "".to_owned();
    
    //Check if the the input contains http/https => link else => id.
    if (&input).starts_with("https://") || (&input).starts_with("http://") {
        match Url::parse(&input) {
            Ok(parsed_url) => {
                let hash_query: HashMap<_ _=""> = parsed_url.query_pairs().into_owned().collect();
                video_id = hash_query.get("v").unwrap().as_str().to_owned();
            }
            Err(e) => eprintln!("{:?}", e),
        }
    } else {
        video_id = input;
    }

    let info_url = format!("https://youtube.com/get_video_info?video_id={}&html5=1", video_id);

    match client.get(&info_url).send().unwrap().text() {
        Ok(v) => {
            let resp_map = serde_urlencoded::from_str::>(&v).unwrap()
                as HashMap;
            let json_player_resp =
                json::parse(&resp_map.get::(&"player_response").unwrap() as &str).unwrap();
            let streaming_data = &json_player_resp["streamingData"]["formats"];
            let video_details = &json_player_resp["videoDetails"];
            println!("video_details: {}", &video_details["title"].to_string());
            let file_name = format!("{}.mp4", &video_details["title"].to_string());
            let normalized_file_name: String = format!(
                "{}",
                file_name
                    .chars()
                    .map(|x| match x {
                        ':' => '-',
                        '"' => ' ',
                        '/' => ' ',
                        '\\' => ' ',
                        '*' => ' ',
                        '?' => ' ',
                        '<' => ' ',
                        '>' => ' ',
                        '|' => ' ',
                        _ => x,
                    })
                    .collect::()
            );
            println!("normalized_file_name: {}", &normalized_file_name);
            if std::path::Path::new(&normalized_file_name).exists() {
                println!("file exist");
                return;
            }
            let mut file = File::create(&normalized_file_name).unwrap();
            match (&streaming_data)[streaming_data.len() - 1]["url"].as_str() {
                Some(url) => {
                    println!("streaming_data: {:?}", &url);
                    let downloaded_video = client.get(url).send().unwrap().bytes().unwrap();
                    file.write_all(&downloaded_video).unwrap();
                }
                None => {
                    download_ciphered_video(&normalized_file_name, &video_id);
                }
            }
        }
        Err(e) => println!("error parsing header: {:?}", e),
    }
}


//Function to use node package ytdl to download ciphered video link.
fn download_ciphered_video(normalize_file_name: &str, video_id: &str) {
    Command::new("node")
        .arg("node_modules/ytdl/bin/ytdl.js")
        .arg(video_id)
        .arg("-o")
        .arg(normalize_file_name)
        .arg("-q")
        .arg("22")
        .output()
        .expect("failed to execute process");
}


The above code will: 

  1. Receive input as video link or video id. 
  2. Fetch the information from info_url "https://youtube.com/get_video_info?video_id=<video_id>"
  3. Choose to download .mp4 format to a file with file name being extracted from the title of the video. If the file is already exist then the program will exit without doing anything.
  4. This project will only handle the easy path to download youtube video. For the more complicated case, I am using node-ytdl. Therefore, there is a need to run npm install ytdl to the folder where the executable program built.
  5. Run test project npm install ytdl && cargo run -- https://www.youtube.com/watch?v=EToeZxIPdKg

This is my "hello world" project in rust. I will need to learn more about idiomatic rust. So far, my feeling working with rust is that it is strict and easy. It is easy because the compiler print very details error messages so that I can search and fix those errors. The downside, in my opinion, is the very slow compile duration. I think rust is fun and worth learning more.


Reference:
node youtube download https://github.com/fent/node-ytdl

- ninjahoahong

Comments

Popular posts from this blog

My books review: "The retrospective handbook"

Overview: The retrospective handbook is a useful collection of tips for preparing and running retrospective for agile teams. The central point of facilitating a retrospective is the preparation. All the details - even the smallest details such as how much paper is available or what the colours of the using pens - play an important role in running a successful retrospective. In such preparation, the most important preparation is the relationship. It requires team members to have high trust with each other to address any problems in great details and find the most suitable solution for those problems. My impression: The tips for running remote retrospective would be really useful for anyone especially startups in my opinion. All the startups I have worked for so far have had some foreign teams either external or internal to save operating money. In addition, there are a lot of remote work to save office renting space and almost all of those startups followed the agile method.

My books review: Money for the rest of us

Overview: The "Money for the rest of us" book is a list of 10 questions serve as a guideline to invest. It is light book with not many pages and can be read during commute time. The book does not require audiences to have any special knowledge. My impression: The author, Mr. David, is very friendly and patient to answer my questions. The 10 questions are greatly explained by the author, and I think they provide a good starting point. However, I think anyone who wants to invest should build their own checklist and a general guideline for different situations. When a general guideline is in place, the decisions need to respect the guideline in order to make consistent decision. Consistency , in my opinion, is the most important in the long run as it provide information for constant feedback and improvements. Overall, I think this is a good book that helps improving my decision making process. - ninjahoahong