Commit Graph

217 Commits (master)

Author SHA1 Message Date
Sunshine 15d98a7269
don't modify base url by default, add option for setting it 3 years ago
Sunshine 38a6f963ad
account for network errors, add option to ignore them 4 years ago
zfhrp a296531b3f remove unused import opts::Options; 4 years ago
Sunshine 1d6392cb28
implement support for BASE tag 4 years ago
Sunshine 03cdc0e0b2
Merge pull request #201 from snshn/refactor-and-version-bump
Refactor and version bump
4 years ago
Sunshine 73c35eaccb
treat minus for output target file path as stdout 4 years ago
Sunshine 90f7c3a0d0
alphabetical order for function names 4 years ago
Sunshine 09d41d2cf1
automatically obtain favicon.ico 4 years ago
Sunshine 164e728ad3
Merge pull request #197 from snshn/addetional-black-box-test-data
Additional black box test data
4 years ago
Sunshine 8883bd6aca
add more black box test data 4 years ago
Sunshine ec85121d28
update help message 4 years ago
Sunshine decd5b2119
add ASCII logo atop of help message 4 years ago
Sunshine 23de5ced21
add tests for utils::indent() 4 years ago
Sunshine bc98aca2a2
indent items in retrieval log to form a tree-like structure 4 years ago
Sunshine 438ebd520a
Merge pull request #193 from snshn/options-struct
Pass options object instead of using separate parameters
4 years ago
Sunshine ddb97009e9
pass options object instead of using separate parameters 4 years ago
Sunshine 9e5d8ec691
add more sample data for blackbox tests 4 years ago
Sunshine 29bf042da0
trim CSS if it contains nothing but whitespaces 4 years ago
Sunshine 2ac964fae5
include font-src into CSP 4 years ago
Sunshine ae5d6d2df4
refactor CSP code 4 years ago
Sunshine 44cac65a83
automatically remove "Refresh" and "Location" META tags 4 years ago
Sunshine 0347fd3985
move metadata tag code into a function 4 years ago
Sunshine 5c229c51da
move functions related to URL manipulation into url.rs 4 years ago
Sunshine f6ea16b3ad
create a separate function for appending URL fragments 4 years ago
Sunshine f9aac6f41b
update crates 4 years ago
Sunshine 4e31d0433e
only attempt to remove credentals from HTTP(S) URLs 4 years ago
Sunshine 090d647390
group all tests into either passing or failing groups 4 years ago
Sunshine cbe3f9f554
implement support for embedding images within srcset 4 years ago
Sunshine a18df74946
refactor code and implement integrity validation 4 years ago
Sunshine c4569343a4
improve metadata comments 4 years ago
Sunshine 5f5820c71a
Merge pull request #168 from snshn/context-comment
Metadata comment tag
4 years ago
Sunshine c999359b9f
Merge branch 'context-comment' of github.com:Alch-Emi/monolith into context-comment 4 years ago
Sunshine f22e2b6e68
embed SVG IMAGE assets 4 years ago
Sunshine 9b40dbbf27
add option to exclude web fonts 4 years ago
Sunshine edacd09dc8
store blobs instead of data URLs in cache 4 years ago
Sunshine eeea617fb1
escape all special chars within #id and .class CSS selectors 4 years ago
Sunshine 9d3df2cdc6
escape colons within CSS idents 4 years ago
Sunshine 3738be2b6d
treat url()'s found in @counter-style rules as images 4 years ago
Sunshine 594ad55bd8
improve data URL media type detection 4 years ago
Sunshine c097733ae7
improve parsing of data URLs 4 years ago
Sunshine 349c7bb3ea
properly parse negative units in CSS 4 years ago
Sunshine a19aa37ea8
Merge pull request #145 from snshn/no-images-svg
Empty SVG nodes when excluding images
4 years ago
Sunshine 791e44796e
resolve hrefs of <area> image-map tags 4 years ago
Sunshine b88479446c
implement unit tests for macros 4 years ago
Sunshine 1d6217ef5a
empty SVG nodes if --no-images 4 years ago
Sunshine 29836d979a
add support for image inputs 4 years ago
Sunshine 643c4ce7ef
implement improvements suggested by @robatipoor 4 years ago
Sunshine 875481b9a2
update help dialog 4 years ago
Sunshine 4951fea730
implement full CSS parsing 4 years ago
Sunshine be25784297
improve SVG media type detection 4 years ago
Sunshine 479c42e1ce
improve test code structure 4 years ago
Sunshine 933379c798
ensure consistent naming across all tests 4 years ago
Sunshine 59a8be493d
add support for working with local assets 4 years ago
Sunshine d5ee8ae6ab
account for legacy BODY background="" attribute 4 years ago
Sunshine 7654eec7e2
treat frames the same way as iframes 4 years ago
Sunshine 0d1e21e9ad
add black box tests 4 years ago
Sunshine 3d2d40e7cd
add support for data URL targets 4 years ago
Sunshine b8b6d8cff6
fix "succeeding" to "passing" in tests 4 years ago
Sunshine 928664dc88
correct is_valid_url to is_http_url 4 years ago
Sunshine 5c8d75539b
rename dataurl to data_url 4 years ago
Sunshine d9602e25eb
update help dialog and README.md 4 years ago
Sunshine f720fe0176
Merge pull request #114 from snshn/custom-network-timeout-option
Add option for custom network request timeout
4 years ago
Sunshine 727a5a410c
add option for custom network request timeout 4 years ago
Sunshine 5ef2b7c9dc
nullify JS within As' href attributes when needed 4 years ago
rhysd 0e12cecd85 Revert "Merge pull request #106 from rhysd/proxy-support"
This reverts commit d8def879b2, reversing
changes made to a9d114d04d.
4 years ago
rhysd d47482fcd9 fix crash at setting empty values to HTTP proxies
with this patch `https_proxy=` and `http_proxy=` will work well.
4 years ago
rhysd b68624f2f3 support HTTP and HTTPS proxies (fix #103) 4 years ago
Sunshine a9d114d04d
Merge pull request #105 from rhysd/refactor-main
Refactoring for main.rs to address several issues
4 years ago
rhysd 4e4ebe9c98 refactor main to address several issues
Addressed issues:

- when specified URL is invalid, it exited successfully with doing
  nothing. There was no way why it does not work for users
- it exited successfully even if invalid User-Agent value is specified
- it created file twice on `--output` option specified. It may cause an
  issue when some file watcher (e.g. FsEvents on macOS) is watching

Improvements:
- handle errors with `Result::expect` consistently it correctly exits
  with non-zero status on error
- define `Output` enum for handling both stdout and file outputs
4 years ago
rhysd 1779f4a374 better comments for JS_DOM_EVENT_ATTRS constant 4 years ago
rhysd 26e89ae6d3 use complete list of DOM event handlers 4 years ago
rhysd 69d99b69e8 remove . in line comment 4 years ago
Emi Simpson 05985583f0
Switch timestamps from rfc822 local time to iso8601 UTC 4 years ago
Emi Simpson 651fa716b4
Clean user, pass, and fragment from URL before writing 4 years ago
rhysd 67b79e92f9 simplify &x.into_iter() to x.iter() 4 years ago
rhysd b51f41fe34 trim attribute values 4 years ago
rhysd 6f158dc6db compare value of 'rel' properties in case-insensitive 4 years ago
rhysd 8d7052b39c ignore preload and prefetch sources
since all resources are embedded as data URL.
4 years ago
rhysd 660511b8a0 define link type of <link> element as enum and prefer match statement
since match statement checks exhaustiveness
4 years ago
Emi Simpson 9be3982dc6
Added --no-context flag to disable adding context comment 4 years ago
Emi Simpson 27c9fb4cd3
Added comment indicating the context under which the page was downloaded 4 years ago
rhysd 6e99ad13e7 upgrade reqwest to v0.10.0
This will improve build time and binary size as follows:

* Before

- **Compile targets**: 220
- **Build time**: `cargo build --release  1264.95s user 39.72s system 335% cpu 6:29.14 total`
- **Binary size**: 6578568 bytes

* After

- **Compile targets**: 170
- **Build time**: `cargo build --release  1130.64s user 32.15s system 359% cpu 5:23.69 total`
- **Binary size**: 6107088 bytes

* Differences

- **Compile targets**: 1.29x smaller
- **Build time**: 1.23x faster
- **Binary size**: 1.07x smaller
4 years ago
Sunshine 413dd66886
Merge pull request #96 from rhysd/refactorings
Refactorings
4 years ago
rhysd dc7ec6e7a8 remove more redundant type annotations 4 years ago
rhysd ed879231af fix test code was broken by refactoring 4 years ago
rhysd ddf4b8ac13 prefer &str to String for reducing allocations 4 years ago
rhysd 84c13f0605 prefer unwrap_or_default to unwrap_or 4 years ago
rhysd ce03e0e487 reduce allocation on checking DOM attributes and do not hard-code number of elements of array constant
`to_lower` allocates new string but the allocation is not necessary
here.
4 years ago
rhysd 63e19998d0 reduce clones and fix some code styles and redundant code 4 years ago
rhysd 75fb6961ed migrate to Rust 2018 4 years ago
Sunshine 5ba8931502
Merge pull request #92 from snshn/output-file-option
Add option for saving output to file
4 years ago
Sunshine 88ffde0c3b wipe integrity attributes 4 years ago
Sunshine bfb97bd062 add option for saving output to file 4 years ago
Sunshine 295931041c
Merge pull request #80 from Alch-Emi/lazyload
Add support for lazy loaded images
4 years ago
Emi Simpson dab4ae6965
Merged Y2Z/master with Alch-Emi/lazyload 4 years ago
Sunshine c7fc121c7c use clean URLs as hashmap keys 4 years ago
Sunshine 9ff9dd0928
Merge pull request #82 from snshn/str
implement str!() macro
4 years ago
Emi Simpson 3d4a932ac1
Merge Y2Z/master, fix conflicts between shared-client & resolve-css 4 years ago
Sunshine 9fe913d853 implement str!() macro 5 years ago
Sunshine 862489e41b Get rid of brackets around URLs 5 years ago