Add data stream progress events and file rendering#30
Draft
1egoman wants to merge 1 commit into
Draft
Conversation
Add logic to handle rendering data streams with names and/or mime types associated as a special "file" state. I am assuming that these will be relatively large so files are buffered in tempfiles rather than in memory. And then also, add a "save" button which can be used to save files to arbitrary locations for inspection.
1egoman
marked this pull request as ready for review
July 16, 2026 20:58
1egoman
commented
Jul 16, 2026
Comment on lines
+639
to
+661
| /// Streams every chunk of a byte reader into an anonymous temp file, returning the open | ||
| /// handle. The file is unlinked at creation (`tempfile`), so the OS reclaims it when the | ||
| /// handle is dropped or the process exits — including on a panic/abort, where `Drop` would | ||
| /// not run (the release profile sets `panic = "abort"`). | ||
| async fn stream_to_temp_file( | ||
| reader: &mut ByteStreamReader, | ||
| entry: &Arc<Mutex<TopicEntry>>, | ||
| n: u64, | ||
| total: Option<u64>, | ||
| ) -> Result<std::fs::File, String> { | ||
| use tokio::io::AsyncWriteExt; | ||
| let std_file = tempfile::tempfile_in(std::env::temp_dir()).map_err(|e| e.to_string())?; | ||
| let mut file = tokio::fs::File::from_std(std_file); | ||
| let mut processed = 0u64; | ||
| while let Some(chunk) = reader.next().await { | ||
| let chunk = chunk.map_err(|e| e.to_string())?; | ||
| file.write_all(&chunk).await.map_err(|e| e.to_string())?; | ||
| processed += chunk.len() as u64; | ||
| update_progress(entry, n, processed, total); | ||
| } | ||
| file.flush().await.map_err(|e| e.to_string())?; | ||
| Ok(file.into_std().await) | ||
| } |
Contributor
Author
There was a problem hiding this comment.
Note to any reviewers - this maybe could make sense as an extension to ByteStreamReader?
1egoman
requested review from
MaxHeimbrock and
xianshijing-lk
and removed request for
xianshijing-lk
July 16, 2026 21:00
1egoman
marked this pull request as draft
July 16, 2026 21:01
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Warning
This change relies on the rust data streams v2 implementation. This cannot be merged until that is merged first. This is also why CI is failing.
Add logic to handle rendering data streams with names and/or mime types associated as a special "file" state. I am assuming that these will be relatively large so files are buffered in tempfiles rather than in memory. And then also, add a "save" button which can be used to save files to arbitrary locations for inspection.
Untitled.mov
I will say, design wise, I think this side panel is starting to break down a little bit now that there is a card rendered within a card. I can try some more stuff here but also I'm not an
eguiexpert so I would be interested in some ideas before investigating anything.