Module forest_filecoin::utils::io::progress_log

source ·
Expand description

It can often take time to perform some operations in Forest and we would like to have a way for logging progress.

Previously we used progress bars thanks to the indicatif library but we had a few issues with them:

  • They behaved poorly together with regular logging
  • They were too verbose and printed even for very small tasks (less than 5 seconds)
  • They were only used when connected to a TTY and not written in log files

This lead us to develop our own logging code. This module provides two new types for logging progress that are WithProgress and WithProgressRaw. The main goal of WithProgressRaw is to maintain a similar API to the previous one from progress bar so we could remove the indicatif dependency, but, gradually, we would like to move to something better and use the WithProgress type. The WithProgress type will provide a way to wrap user code while handling logging presentation details. WithProgress is a wrapper that should extend to Iterators, Streams, Read/Write types. Right now it only wraps async reads.

§Example

use tokio_test::block_on;
use tokio::io::AsyncBufReadExt;
use forest_filecoin::doctest_private::WithProgress;
block_on(async {
    let data: String = "some very big string".into();
    let mut reader = tokio::io::BufReader::new(data.as_bytes());
    let len = 0; // Compute total read length or find of way to estimate it
    // We just need to wrap our reader and use the wrapped version
    let reader_wp = tokio::io::BufReader::new(WithProgress::wrap_async_read("reading", reader, len));
    let mut stream = reader_wp.lines();
    while let Some(line) = stream.next_line().await.unwrap() {
        // Do something with the line
    }
})

§Future work

  • Add and move progressively to new API (Iterator, Streams), and removed deprecated usage of WithProgressRaw
  • Add a more accurate ETA etc

Structs§

Enums§

Constants§