pax_global_header00006660000000000000000000000064146420027230014512gustar00rootroot0000000000000052 comment=56b54dd8db056a416d0d42456c00b239e45cb807 firecat53-urlscan-03f47eb/000077500000000000000000000000001464200272300153605ustar00rootroot00000000000000firecat53-urlscan-03f47eb/.github/000077500000000000000000000000001464200272300167205ustar00rootroot00000000000000firecat53-urlscan-03f47eb/.github/workflows/000077500000000000000000000000001464200272300207555ustar00rootroot00000000000000firecat53-urlscan-03f47eb/.github/workflows/main.yml000066400000000000000000000012001464200272300224150ustar00rootroot00000000000000name: main on: push: branches: - main pull_request: branches: - main jobs: build: runs-on: ubuntu-latest strategy: fail-fast: false matrix: python-version: ['3.7', '3.8', '3.9', '3.10', '3.11', '3.12'] steps: - uses: actions/checkout@v4 - name: Set up Python ${{ matrix.python-version }} uses: actions/setup-python@v5 with: python-version: ${{ matrix.python-version }} - name: Install Hatch run: | python -m pip install --upgrade pip pip install hatch - name: Build package run: hatch build firecat53-urlscan-03f47eb/.gitignore000066400000000000000000000001261464200272300173470ustar00rootroot00000000000000*.pyc *.egg *.egg-info *.gz *.xz build dist test_emails/ urlscan/_version.py MANIFEST firecat53-urlscan-03f47eb/LICENSE000066400000000000000000000431031464200272300163660ustar00rootroot00000000000000 GNU GENERAL PUBLIC LICENSE Version 2, June 1991 Copyright (C) 1989, 1991 Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. Preamble The licenses for most software are designed to take away your freedom to share and change it. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change free software--to make sure the software is free for all its users. This General Public License applies to most of the Free Software Foundation's software and to any other program whose authors commit to using it. (Some other Free Software Foundation software is covered by the GNU Lesser General Public License instead.) You can apply it to your programs, too. When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for this service if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs; and that you know you can do these things. To protect your rights, we need to make restrictions that forbid anyone to deny you these rights or to ask you to surrender the rights. These restrictions translate to certain responsibilities for you if you distribute copies of the software, or if you modify it. For example, if you distribute copies of such a program, whether gratis or for a fee, you must give the recipients all the rights that you have. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights. We protect your rights with two steps: (1) copyright the software, and (2) offer you this license which gives you legal permission to copy, distribute and/or modify the software. Also, for each author's protection and ours, we want to make certain that everyone understands that there is no warranty for this free software. If the software is modified by someone else and passed on, we want its recipients to know that what they have is not the original, so that any problems introduced by others will not reflect on the original authors' reputations. Finally, any free program is threatened constantly by software patents. We wish to avoid the danger that redistributors of a free program will individually obtain patent licenses, in effect making the program proprietary. To prevent this, we have made it clear that any patent must be licensed for everyone's free use or not licensed at all. The precise terms and conditions for copying, distribution and modification follow. GNU GENERAL PUBLIC LICENSE TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 0. This License applies to any program or other work which contains a notice placed by the copyright holder saying it may be distributed under the terms of this General Public License. The "Program", below, refers to any such program or work, and a "work based on the Program" means either the Program or any derivative work under copyright law: that is to say, a work containing the Program or a portion of it, either verbatim or with modifications and/or translated into another language. (Hereinafter, translation is included without limitation in the term "modification".) Each licensee is addressed as "you". Activities other than copying, distribution and modification are not covered by this License; they are outside its scope. The act of running the Program is not restricted, and the output from the Program is covered only if its contents constitute a work based on the Program (independent of having been made by running the Program). Whether that is true depends on what the Program does. 1. You may copy and distribute verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice and disclaimer of warranty; keep intact all the notices that refer to this License and to the absence of any warranty; and give any other recipients of the Program a copy of this License along with the Program. You may charge a fee for the physical act of transferring a copy, and you may at your option offer warranty protection in exchange for a fee. 2. You may modify your copy or copies of the Program or any portion of it, thus forming a work based on the Program, and copy and distribute such modifications or work under the terms of Section 1 above, provided that you also meet all of these conditions: a) You must cause the modified files to carry prominent notices stating that you changed the files and the date of any change. b) You must cause any work that you distribute or publish, that in whole or in part contains or is derived from the Program or any part thereof, to be licensed as a whole at no charge to all third parties under the terms of this License. c) If the modified program normally reads commands interactively when run, you must cause it, when started running for such interactive use in the most ordinary way, to print or display an announcement including an appropriate copyright notice and a notice that there is no warranty (or else, saying that you provide a warranty) and that users may redistribute the program under these conditions, and telling the user how to view a copy of this License. (Exception: if the Program itself is interactive but does not normally print such an announcement, your work based on the Program is not required to print an announcement.) These requirements apply to the modified work as a whole. If identifiable sections of that work are not derived from the Program, and can be reasonably considered independent and separate works in themselves, then this License, and its terms, do not apply to those sections when you distribute them as separate works. But when you distribute the same sections as part of a whole which is a work based on the Program, the distribution of the whole must be on the terms of this License, whose permissions for other licensees extend to the entire whole, and thus to each and every part regardless of who wrote it. Thus, it is not the intent of this section to claim rights or contest your rights to work written entirely by you; rather, the intent is to exercise the right to control the distribution of derivative or collective works based on the Program. In addition, mere aggregation of another work not based on the Program with the Program (or with a work based on the Program) on a volume of a storage or distribution medium does not bring the other work under the scope of this License. 3. You may copy and distribute the Program (or a work based on it, under Section 2) in object code or executable form under the terms of Sections 1 and 2 above provided that you also do one of the following: a) Accompany it with the complete corresponding machine-readable source code, which must be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, b) Accompany it with a written offer, valid for at least three years, to give any third party, for a charge no more than your cost of physically performing source distribution, a complete machine-readable copy of the corresponding source code, to be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, c) Accompany it with the information you received as to the offer to distribute corresponding source code. (This alternative is allowed only for noncommercial distribution and only if you received the program in object code or executable form with such an offer, in accord with Subsection b above.) The source code for a work means the preferred form of the work for making modifications to it. For an executable work, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the executable. However, as a special exception, the source code distributed need not include anything that is normally distributed (in either source or binary form) with the major components (compiler, kernel, and so on) of the operating system on which the executable runs, unless that component itself accompanies the executable. If distribution of executable or object code is made by offering access to copy from a designated place, then offering equivalent access to copy the source code from the same place counts as distribution of the source code, even though third parties are not compelled to copy the source along with the object code. 4. You may not copy, modify, sublicense, or distribute the Program except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense or distribute the Program is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance. 5. You are not required to accept this License, since you have not signed it. However, nothing else grants you permission to modify or distribute the Program or its derivative works. These actions are prohibited by law if you do not accept this License. Therefore, by modifying or distributing the Program (or any work based on the Program), you indicate your acceptance of this License to do so, and all its terms and conditions for copying, distributing or modifying the Program or works based on it. 6. Each time you redistribute the Program (or any work based on the Program), the recipient automatically receives a license from the original licensor to copy, distribute or modify the Program subject to these terms and conditions. You may not impose any further restrictions on the recipients' exercise of the rights granted herein. You are not responsible for enforcing compliance by third parties to this License. 7. If, as a consequence of a court judgment or allegation of patent infringement or for any other reason (not limited to patent issues), conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot distribute so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not distribute the Program at all. For example, if a patent license would not permit royalty-free redistribution of the Program by all those who receive copies directly or indirectly through you, then the only way you could satisfy both it and this License would be to refrain entirely from distribution of the Program. If any portion of this section is held invalid or unenforceable under any particular circumstance, the balance of the section is intended to apply and the section as a whole is intended to apply in other circumstances. It is not the purpose of this section to induce you to infringe any patents or other property right claims or to contest validity of any such claims; this section has the sole purpose of protecting the integrity of the free software distribution system, which is implemented by public license practices. Many people have made generous contributions to the wide range of software distributed through that system in reliance on consistent application of that system; it is up to the author/donor to decide if he or she is willing to distribute software through any other system and a licensee cannot impose that choice. This section is intended to make thoroughly clear what is believed to be a consequence of the rest of this License. 8. If the distribution and/or use of the Program is restricted in certain countries either by patents or by copyrighted interfaces, the original copyright holder who places the Program under this License may add an explicit geographical distribution limitation excluding those countries, so that distribution is permitted only in or among countries not thus excluded. In such case, this License incorporates the limitation as if written in the body of this License. 9. The Free Software Foundation may publish revised and/or new versions of the General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. Each version is given a distinguishing version number. If the Program specifies a version number of this License which applies to it and "any later version", you have the option of following the terms and conditions either of that version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of this License, you may choose any version ever published by the Free Software Foundation. 10. If you wish to incorporate parts of the Program into other free programs whose distribution conditions are different, write to the author to ask for permission. For software which is copyrighted by the Free Software Foundation, write to the Free Software Foundation; we sometimes make exceptions for this. Our decision will be guided by the two goals of preserving the free status of all derivatives of our free software and of promoting the sharing and reuse of software generally. NO WARRANTY 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. END OF TERMS AND CONDITIONS How to Apply These Terms to Your New Programs If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms. To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively convey the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found. Copyright (C) This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. Also add information on how to contact you by electronic and paper mail. If the program is interactive, make it output a short notice like this when it starts in an interactive mode: Gnomovision version 69, Copyright (C) year name of author Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. This is free software, and you are welcome to redistribute it under certain conditions; type `show c' for details. The hypothetical commands `show w' and `show c' should show the appropriate parts of the General Public License. Of course, the commands you use may be called something other than `show w' and `show c'; they could even be mouse-clicks or menu items--whatever suits your program. You should also get your employer (if you work as a programmer) or your school, if any, to sign a "copyright disclaimer" for the program, if necessary. Here is a sample; alter the names: Yoyodyne, Inc., hereby disclaims all copyright interest in the program `Gnomovision' (which makes passes at compilers) written by James Hacker. , 1 April 1989 Ty Coon, President of Vice This General Public License does not permit incorporating your program into proprietary programs. If your program is a subroutine library, you may consider it more useful to permit linking proprietary applications with the library. If this is what you want to do, use the GNU Lesser General Public License instead of this License. firecat53-urlscan-03f47eb/README.md000066400000000000000000000211671464200272300166460ustar00rootroot00000000000000# Urlscan [![main](https://github.com/firecat53/urlscan/actions/workflows/main.yml/badge.svg)](https://github.com/firecat53/urlscan/actions/workflows/main.yml) ## Contributors Scott Hansen \ (Author and Maintainer) Maxime Chatelle \ (Debian Maintainer) Daniel Burrows \ (Original Author) ## Purpose and Requirements Urlscan is a small program that is designed to integrate with the "mutt" mailreader to allow you to easily launch a Web browser for URLs contained in email messages. It is a replacement for the "urlview" program. Requires: Python 3.7+ and the python-urwid library ## Features Urlscan parses an email message or file and scans it for URLs and email addresses. It then displays the URLs and their context within the message, and allows you to choose one or more URLs to send to your Web browser. Alternatively, it send a list of all URLs to stdout. Relative to urlview, urlscan has the following additional features: - Support for emails in quoted-printable and base64 encodings. No more stripping out =40D from URLs by hand! - The context of each URL is provided along with the URL. For HTML mails, a crude parser is used to render the HTML into text. Context view can be toggled on/off with `c`. - URLs are shortened by default to fit on one line. Viewing full URL (for one or all) is toggled with `s` or `S`. - Jump to a URL by typing the number. - Incremental case-insensitive search with `/`. - Execute an arbitrary function (for example, copy URL to clipboard) instead of opening URL in a browser. - Use `l` to cycle through whether URLs are opened using the Python webbrowser module (default), xdg-open (if installed) or opened by a function passed on the command line with `--run` or `--run-safe`. - Configure colors and keybindings via ~/.config/urlscan/config.json. Generate default config file for editing by running `urlscan -g`. Cycle through available palettes with `p`. Set display width with `--width`. - Copy URL to clipboard with `C` or to primary selection with `P`. Requires xsel or xclip. - Run a command with the selected URL as the argument or pipe the selected URL to a command. - Show complete help menu with `F1`. Hide header on startup with `--nohelp`. - Use a custom regular expression with `-E` for matching urls or any other pattern. In junction with `-r`, this effectively turns urlscan into a general purpose CLI selector-type utility. - Scan certain email headers for URLs. Currently `Link`, `Archived-At` and `List-*` are scanned when `--headers` is passed. - Queue multiple URLs for opening and open them all at once with `a` and `o`. ## Installation and setup To install urlscan, install from your distribution repositories, from Pypi, or do a local development install with pip -e: pipx install urlscan OR pip install --user urlscan OR cd && pip install --user -e . **NOTE** The minimum required version of urwid is 1.2.1. Once urlscan is installed, add the following lines to your .muttrc: macro index,pager \cb " urlscan" "call urlscan to extract URLs out of a message" macro attach,compose \cb " urlscan" "call urlscan to extract URLs out of a message" Once this is done, Control-b while reading mail in mutt will automatically invoke urlscan on the message. > Note for Neomutt users: [As of version > `2023-05-17`](https://github.com/neomutt/neomutt/releases/tag/20230517) true > color support was implemented. If you are using true color support with Neomutt, > or are encountering the error `setupterm: could not find terminfo database`, > then you should also add `TERM=xterm-256color` to your macro in `.muttrc`. > See more here [#135](https://github.com/firecat53/urlscan/issues/135). For example: > `macro index,pager \cb " TERM=xterm-256color urlscan" "call urlscan to extract URLs out of a message"` To choose a particular browser, set the environment variable BROWSER. If BROWSER is not set, xdg-open will control which browser is used, if it's available.: export BROWSER=/usr/bin/epiphany ## Command Line usage urlscan OPTIONS OPTIONS [-c, --compact] [-d, --dedupe] [-E, --regex ] [-f, --run-safe ] [-g, --genconf] [-H, --nohelp] [ --headers] [-n, --no-browser] [-p, --pipe] [-r, --run ] [-R, --reverse] [-s, --single] [-w, --width] [-W --whitespace-off] Urlscan can extract URLs and email addresses from emails or any text file. Calling with no flags will start the curses browser. Calling with '-n' will just output a list of URLs/email addressess to stdout. The '-c' flag removes the context from around the URLs in the curses browser, and the '-d' flag removes duplicate URLs. The '-R' flag reverses the displayed order of URLs and context. Files can also be piped to urlscan using normal shell pipe mechanisms: `cat | urlscan` or `urlscan < `. The '-W' flag condenses the display output by suppressing blank lines and ellipses lines. Instead of opening a web browser, the selected URL can be passed as the argument to a command using `--run-safe " {}"` or `--run " {}"`. Note the use of `{}` in the command string to denote the selected URL. Alternatively, the URL can be piped to the command using `--run-safe --pipe` (or `--run`). Using --run-safe with --pipe is preferred if the command supports it, as it is marginally more secure and tolerant of special characters in the URL. ## Theming Run `urlscan -g` to generate ~/.config/urlscan/config.json with the default color and black & white palettes. This can be edited or added to, as desired. The first palette in the list will be the default. Configure the palettes according to the [Urwid display attributes][1]. Display width can be set with `--width`. ## Keybindings Run `urlscan -g` to generate ~/.config/urlscan/config.json. All of the keys will be listed. You can either leave in place or delete any that will not be altered. To unset a binding, set it equal to "". For example: `"P": ""` The follow actions are supported: - `add_url` -- add a URL to the queue (default: `a`) - `all_escape` -- toggle unescape all URLs (default: `u`) - `all_shorten` -- toggle shorten all URLs (default: `S`) - `bottom` -- move cursor to last item (default: `G`) - `clear_screen` -- redraw screen (default: `Ctrl-l`) - `clipboard` -- copy highlighted URL to clipboard using xsel/xclip (default: `C`) - `clipboard_pri` -- copy highlighted URL to primary selection using xsel/xclip (default: `P`) - `context` -- show/hide context (default: `c`) - `del_url` -- delete URL from the queue (default: `d`) - `down` -- cursor down (default: `j`) - `help_menu` -- show/hide help menu (default: `F1`) - `link_handler` -- cycle link handling (webbrowser, xdg-open, --run-safe or --run) (default: `l`) - `next` -- jump to next URL (default: `J`) - `open_queue` -- open all URLs in queue (default: `o`) - `open_queue_win` -- open all URLs in queue in new window (default: `O`) - `open_url` -- open selected URL (default: `space` or `enter`) - `palette` -- cycle through palettes (default: `p`) - `previous` -- jump to previous URL (default: `K`) - `quit` -- quit (default: `q` or `Q`) - `reverse` -- reverse display order (default: `R`) - `shorten` -- toggle shorten highlighted URL (default: `s`) - `top` -- move to first list item (default: `g`) - `up` -- cursor up (default: `k`) ## Update TLD list (for developers, not users) `wget https://data.iana.org/TLD/tlds-alpha-by-domain.txt` ## Known bugs and limitations - Running urlscan sometimes "messes up" the terminal background. This seems to be an urwid bug, but I haven't tracked down just what's going on. - Extraction of context from HTML messages leaves something to be desired. Probably the ideal solution would be to extract context on a word basis rather than on a paragraph basis. - The HTML message handling is a bit kludgy in general. - multipart/alternative sections are handled by descending into all the sub-parts, rather than just picking one, which may lead to URLs and context appearing twice. (Bypass this by selecting the '--dedupe' option) ## Build/development - pyproject.toml is configured for [hatch][2] for building and submitting to pypi. - flake.nix is available for a development shell or building/testing the package if desired. `nix develop` [1]: http://urwid.org/manual/displayattributes.html#display-attributes "Urwid display attributes" [2]: https://hatch.pypa.io/latest/ "Hatch" firecat53-urlscan-03f47eb/flake.lock000066400000000000000000000010261464200272300173130ustar00rootroot00000000000000{ "nodes": { "nixpkgs": { "locked": { "lastModified": 1717298511, "narHash": "sha256-9sXuJn/nL+9ImeYtlspTvjt83z1wIgU+9AwfNbnq+tI=", "owner": "NixOS", "repo": "nixpkgs", "rev": "6634a0509e9e81e980b129435fbbec518ab246d0", "type": "github" }, "original": { "owner": "NixOS", "repo": "nixpkgs", "type": "github" } }, "root": { "inputs": { "nixpkgs": "nixpkgs" } } }, "root": "root", "version": 7 } firecat53-urlscan-03f47eb/flake.nix000066400000000000000000000032551464200272300171670ustar00rootroot00000000000000{ description = "View/select the URLs in an email message or file"; inputs = { nixpkgs.url = "github:NixOS/nixpkgs"; }; outputs = { self, nixpkgs, }: let systems = ["x86_64-linux" "i686-linux" "aarch64-linux"]; forAllSystems = f: nixpkgs.lib.genAttrs systems (system: f rec { pkgs = nixpkgs.legacyPackages.${system}; commonPackages = builtins.attrValues { inherit (pkgs.python312Packages) python urwid ; }; }); in { devShells = forAllSystems ({ pkgs, commonPackages, }: { default = pkgs.mkShell { packages = commonPackages ++ [pkgs.pandoc]; shellHook = '' alias urlscan="python -m urlscan" export PYTHONPATH="$PYTHONPATH:$PWD" ''; }; }); packages = forAllSystems ({ pkgs, commonPackages, }: { default = pkgs.python312Packages.buildPythonApplication { name = "urlscan"; pname = "urlscan"; format = "pyproject"; src = ./.; nativeBuildInputs = builtins.attrValues { inherit (pkgs) git ; inherit (pkgs.python312Packages) hatchling hatch-vcs ; }; propagatedBuildInputs = commonPackages; meta = { description = "View/select the URLs in an email message or file"; homepage = "https://github.com/firecat53/urlscan"; license = pkgs.lib.licenses.gpl2Plus; maintainers = ["firecat53"]; platforms = systems; }; }; }); }; } firecat53-urlscan-03f47eb/pyproject.toml000066400000000000000000000030021464200272300202670ustar00rootroot00000000000000[build-system] requires = ["hatchling", "hatch-vcs"] build-backend = "hatchling.build" [project] name = "urlscan" dynamic = ["version"] description = "View/select the URLs in an email message or file" readme = "README.md" license = "GPL-2.0-or-later" authors = [ { name = "Scott Hansen", email = "tech@firecat53.net" }, ] keywords = [ "email", "mutt", "tmux", "urlscan", "urlview", ] classifiers = [ "Development Status :: 5 - Production/Stable", "Environment :: Console", "Environment :: Console :: Curses", "License :: OSI Approved :: GNU General Public License v2 (GPLv2)", "Operating System :: OS Independent", "Programming Language :: Python", "Programming Language :: Python :: 3.7", "Programming Language :: Python :: 3.8", "Programming Language :: Python :: 3.9", "Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", "Topic :: Utilities", ] dependencies = [ "urwid>=1.2.1", ] [project.scripts] urlscan = "urlscan.__main__:main" [project.urls] Homepage = "https://github.com/firecat53/urlscan" [tool.hatch.version] source = "vcs" fallback-version = "0.0.0" [tool.hatch.build.hooks.vcs] version-file = "urlscan/_version.py" [tool.hatch.build.targets.wheel.shared-data] LICENSE = "share/doc/urlscan/LICENSE" "README.md" = "share/doc/urlscan/README.md" "urlscan.1" = "share/man/man1/urlscan.1" [tool.hatch.build.targets.sdist] include = [ "/urlscan", "urlscan.1", ] firecat53-urlscan-03f47eb/requirements.txt000066400000000000000000000000151464200272300206400ustar00rootroot00000000000000urwid>=1.2.1 firecat53-urlscan-03f47eb/urlscan.1000066400000000000000000000164211464200272300171150ustar00rootroot00000000000000.\" Hey, EMACS: -*- nroff -*- .TH URLSCAN 1 "1 June 2024" .SH NAME urlscan \- browse the URLs in an email message from a terminal .SH SYNOPSIS \fBurlscan\fR [options] < .I message \fBurlscan\fR [options] .I message .SH DESCRIPTION \fBurlscan\fR accepts a single email message on standard input, then displays a terminal-based list of the URLs in the given message. Selecting a URL uses the Python webbrowser module to determine which browser to open. The \fBBROWSER\fR environment variable will be used if it is set. \fBurlscan\fR is primarily intended to be used with the .B mutt (1) mailreader, but it should work well with any terminal-based mail program. \fBurlscan\fR is similar to \fBurlview\fR(1), but has the following additional features: \fB1.\fR Support for more message encodings, such as quoted-printable and base64. \fB2.\fR Extraction and display of the context surrounding each URL. Toggle context view on/off with \fBc\fR. Reverse displayed order of URLs with \fBR\fR. \fB3.\fR Copy current URL to primary selection with \fBP\fR or to clipboard with \fBC\fR. \fB4.\fR URLs are shortened by default to fit on one line. Toggle one or all shortened URLs with \fBs\fR or \fBS\fR. \fB5.\fR Incremental case-insensitive search using \fB/\fR. Footer shows current search term. \fB/\fR again resets search. \fB6.\fR Cycle through all available palettes (color and black & white available by default) using \fBp\fR. Running \fBurlscan \-g\fR will generate a ~/.config/urlscan/config.json file for editing or adding additional pallettes and keybindings. See http://urwid.org/manual/displayattributes.html#display-attributes for color options and allowed values. Set display width with `--width`. \fB7.\fR \fBu\fR will unescape the highlighted URL if necessary. \fB8.\fR Run a command with the selected URL as the argument or pipe the selected URL to a command using the \fB--run-safe\fR, \fB--run\fR and \fB--pipe\fR arguments. \fB9.\fR Use \fBl\fR to cycle through whether URLs are opened using the Python webbrowser module (default), xdg-open (if installed) or a function passed on the command line with \fB--run-safe\fR or \fB--run\fR. The \fB--run\fR and \fB--run-safe\fR functions will respect the value of \fB--pipe\fR. \fB10.\fR \fBF1\fR shows the help menu. \fB11.\fR Scan certain email headers for URLs. Currently \fBLink\fR, \fBArchived-At\fR and \fBList-*\fR are scanned when \fB--headers\fR is passed. \fB12.\fR Queue multiple URLs for opening and open them all at once with \fBa\fR and \fBo\fR. .SH OPTIONS .TP .B \-c, \-\-compact Display a simple list of the extracted URLs, instead of showing the context of each URL. Also toggle with `c` from within the viewer. .TP .B \-d, \-\-dedupe Remove duplicated URLs from the list of URLs. .TP .B \-E, \-\-regex \ Use \ in place of the default set of regular expressions, to be used for any kind of matching. This is useful for example when selectively avoiding 'mailto:' links or any other pattern that urlscan could interpret as urls (such as '.'). Usage example: $ urlscan --regex 'https?://.+\.\w+' file.txt .TP .B \-f, \-\-run\-safe \ Execute \ in place of opening URL with a browser. Use {} in \ to substitute in the URL. Examples: $ urlscan --run-safe 'tmux set buffer {}' .TP .B \-g, \-\-genconf Generate ~/.config/urlscan/config.json with default options. .TP .B \-H, \-\-nohelp Start with header menu hidden. .TP .B \-\-headers Scan email headers for URLs. .TP .B \-n, \-\-no-browser Disables the selection interface and print the links to standard output. Useful for scripting (implies \fB\-\-compact\fR). .TP .B \-p, \-\-pipe Pipe the selected URL to the command specified by `--run-safe` or `--run`. This is preferred when the command supports it, as it is more secure and tolerant of special characters in the URL. Example: $ urlscan --run-safe 'xclip -i' --pipe file.txt .TP .B \-R, \-\-reverse Reverse displayed order of URLs. .TP .B \-r, \-\-run \ Execute \ in place of opening URL with a browser. Use {} in \ to substitute in the URL. Shell features such as | and \> can be used, but it is less secure. Examples: $ urlscan --run 'echo {} | xclip -i' file.txt .TP .B \-s, \-\-single Exit urlscan after opening or copying a single browser link. $ urlscan -s file.txt .TP .B \-w, \-\-width Set display width. .TP .B \-W, \-\-whitespace-off Suppress output of blank lines and ellipses lines. .SH MUTT INTEGRATION To integrate urlscan with mutt, include the following two commands in \fB~/.muttrc\fR: .ad l macro index,pager \\cb " urlscan" "call urlscan to extract URLs out of a message" macro attach,compose \\cb " urlscan" "call urlscan to extract URLs out of a message" .ad b Once these lines are in your mutt configuration file, pressing Control-b will allow you to browse and open the URLs in the currently selected message. Alternately, you can pipe a message into urlscan using the '|' operator. This can be useful for applying a different flag (such as the '-d' or '-c' options). .SH KEYBINDINGS Run \fBurlscan \-g\fR to generate ~/.config/urlscan/config.json. All of the keys will be listed. You can either leave in place or delete any that will not be altered. To unset a binding, set it equal to "". For example: \fB"P": ""\fR The follow actions are supported: .TP \fBadd_url\fR \-\- add a URL to the queue (Default: \fBa\fR) .TP \fBall_escape\fR \-\- toggle unescape all URLs (Default: \fBu\fR) .TP \fBall_shorten\fR \-\- toggle shorten all URLs (Default: \fBS\fR) .TP \fBbottom\fR \-\- move cursor to last item (Default: \fBG\fR) .TP \fBclear_screen\fR \-\- redraw screen (Default: \fBCtrl-l\fR) .TP \fBclipboard\fR \-\- copy highlighted URL to clipboard using xsel/xclip (Default: \fBC\fR) .TP \fBclipboard_pri\fR \-\- copy highlighted URL to primary selection using xsel/xclip (Default: \fBP\fR) .TP \fBcontext\fR \-\- show/hide context (Default: \fBc\fR) .TP \fBdel_url\fR \-\- delete URL from the queue (Default: \fBd\fR) .TP \fBdown\fR \-\- cursor down (Default: \fBj\fR) .TP \fBhelp_menu\fR \-\- show/hide help menu (Default: \fBF1\fR) .TP \fBlink_handler\fR \-\- cycle link handling (webbrowser, xdg-open or custom) (Default: \fBl\fR) .TP \fBnext\fR \-\- jump to next URL (Default: \fBJ\fR) .TP \fBopen_queue\fR \-\- open all URLs in queue (Default: \fBo\fR) .TP \fBopen_queue_win\fR \-\- open all URLs in queue in new window (Default: \fBO\fR) .TP \fBopen_url\fR \-\- open selected URL (Default: \fBspace\fR or \fBenter\fR) .TP \fBpalette\fR \-\- cycle through palettes (Default: \fBp\fR) .TP \fBprevious\fR \-\- jump to previous URL (Default: \fBK\fR) .TP \fBquit\fR \-\- quit (Default: \fBq\fR or \fBQ\fR) .TP \fBreverse\fR \-\- reverse display order (Default: \fBR\fR) .TP \fBshorten\fR \-\- toggle shorten highlighted URL (Default: \fBs\fR) .TP \fBtop\fR \-\- move to first list item (Default: \fBg\fR) .TP \fBup\fR \-\- cursor up (Default: \fBk\fR) .SH FILES $HOME/.config/urlscan/config.json Only required if additional or modified palettes or keybindings are desired. .SH SEE ALSO \fI/usr/share/doc/urlscan/README\fR, \fBurlview\fR(1), \fBmutt\fR(1) .SH AUTHOR This manual page was written by Daniel Burrows and Scott Hansen firecat53-urlscan-03f47eb/urlscan/000077500000000000000000000000001464200272300170275ustar00rootroot00000000000000firecat53-urlscan-03f47eb/urlscan/__init__.py000066400000000000000000000000561464200272300211410ustar00rootroot00000000000000__all__ = ['browser', 'urlchoose', 'urlscan'] firecat53-urlscan-03f47eb/urlscan/__main__.py000066400000000000000000000201511464200272300211200ustar00rootroot00000000000000#!/usr/bin/env python3 """ A simple urlview replacement that handles things like quoted-printable properly. """ # # Copyright (C) 2006-2007 Daniel Burrows # Copyright (C) 2023 Scott Hansen # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA import argparse import io import os import sys from email import policy from email.parser import BytesParser from urlscan import urlchoose, urlscan from urlscan._version import version def parse_arguments(): """Parse command line options. Returns: args """ arg_parse = argparse.ArgumentParser(description="Parse and display URLs") arg_parse.add_argument('--genconf', '-g', action='store_true', default=False, help="Generate config file and exit.") arg_parse.add_argument('--compact', '-c', action='store_true', default=False, help="Don't display the context of each URL.") arg_parse.add_argument('--reverse', '-R', dest="reverse", action='store_true', default=False, help="Reverse order of displayed URLS/context") arg_parse.add_argument('--no-browser', '-n', dest="nobrowser", action='store_true', default=False, help="Pipe URLs to stdout") arg_parse.add_argument('--dedupe', '-d', dest="dedupe", action='store_true', default=False, help="Remove duplicate URLs from list") arg_parse.add_argument('--regex', '-E', help="Alternate custom regex to be used for all " "kinds of matching. " r"For example: --regex 'https?://.+\.\w+'") arg_parse.add_argument('--run', '-r', help="Alternate command to run on selected URL " "instead of opening URL in browser. Use {} to " "represent the URL value in the expression. " "For example: --run 'echo {} | xclip -i'") arg_parse.add_argument('--run-safe', '-f', dest="runsafe", help="Alternate command to run on selected URL " "instead of opening URL in browser. Use {} to " "represent the URL value in the expression. Safest " "run option but uses `shell=False` which does not " "allow use of shell features like | or >. Can use " "with --pipe.") arg_parse.add_argument('--pipe', '-p', dest='pipe', action='store_true', default=False, help="Pipe URL into the command specified by --run or --run-safe") arg_parse.add_argument('--nohelp', '-H', dest='nohelp', action='store_true', default=False, help='Hide help menu by default') arg_parse.add_argument('--single', '-s', dest='single', action='store_true', default=False, help='Quit urlscan after opening/copying a single link.') arg_parse.add_argument('--width', '-w', dest='width', type=int, default=0, help='Set width to display') arg_parse.add_argument('--whitespace-off', '-W', dest='whitespaceoff', action='store_true', default=False, help="Don't display empty lines and ellipses.") arg_parse.add_argument('--headers', dest='headers', action='store_true', default=False, help='Scan certain message headers for URLs.') arg_parse.add_argument('--version', '-V', dest='version', action='store_true', default=False, help='Print urlscan version') arg_parse.add_argument('message', nargs='?', default=sys.stdin, help="Filename of the message to parse") return arg_parse.parse_args() def close_stdin(): """This section closes out sys.stdin if necessary so as not to block curses keyboard inputs """ if not os.isatty(0): try: fdesc = os.open('/dev/tty', os.O_RDONLY) except OSError: # This is most likely a non-interactive session, try to open # `stdin` directly fdesc = os.open('/dev/stdin', os.O_RDONLY) if fdesc < 0: sys.stderr.write('Unable to open an input tty.\n') sys.exit(-1) else: os.dup2(fdesc, 0) os.close(fdesc) def process_input(fname): """Return the parsed text of stdin or the message. Accounts for possible file encoding differences. Args: fname - filename or sys.stdin Returns: mesg - EmailMessage object """ if fname is sys.stdin: try: stdin_file = fname.buffer.read() except AttributeError: stdin_file = fname.read() else: stdin_file = None if stdin_file is not None: fobj = io.BytesIO(stdin_file) else: fobj = io.open(fname, mode='rb') f_keep = fobj mesg = BytesParser(policy=policy.default.clone(utf8=True)).parse(fobj) if 'From' not in mesg.keys() and 'Date' not in mesg.keys(): # If it's not an email message, don't let the email parser # delete the first line. If it is, let the parser do its job so # we don't get mailto: links for all the To and From addresses fobj = _fix_first_line(f_keep) mesg = BytesParser(policy=policy.default.clone(utf8=True)).parse(fobj) try: fobj.close() except NameError: pass close_stdin() return mesg def _fix_first_line(fline): """If the first line starts with http* or [ or other non-text characters, the URLs on that line will not be parsed by email.Parser. Add a blank line at the top of the file to ensure everything is read in a non-email file. """ fline.seek(0) new = io.BytesIO() new.write(b"\n" + fline.read()) fline.close() new.seek(0) return new def main(): """Entrypoint function for urlscan """ args = parse_arguments() if args.version is True: print(version) return if args.genconf is True: urlchoose.URLChooser([], genconf=True) return msg = process_input(args.message) if args.nobrowser is False: tui = urlchoose.URLChooser(urlscan.msgurls(msg, regex=args.regex, headers=args.headers), compact=args.compact, reverse=args.reverse, nohelp=args.nohelp, dedupe=args.dedupe, run=args.run, runsafe=args.runsafe, single=args.single, width=args.width, whitespaceoff=args.whitespaceoff, pipe=args.pipe) tui.main() else: out = urlchoose.URLChooser(urlscan.msgurls(msg, regex=args.regex, headers=args.headers), dedupe=args.dedupe, reverse=args.reverse, shorten=False) if args.reverse is True: out.urls.reverse() print("\n".join(out.urls)) if __name__ == "__main__": main() firecat53-urlscan-03f47eb/urlscan/assets/000077500000000000000000000000001464200272300203315ustar00rootroot00000000000000firecat53-urlscan-03f47eb/urlscan/assets/tlds-alpha-by-domain.txt000066400000000000000000000226331464200272300250060ustar00rootroot00000000000000# Version 2024060100, Last Updated Sat Jun 1 07:07:02 2024 UTC AAA AARP ABB ABBOTT ABBVIE ABC ABLE ABOGADO ABUDHABI AC ACADEMY ACCENTURE ACCOUNTANT ACCOUNTANTS ACO ACTOR AD ADS ADULT AE AEG AERO AETNA AF AFL AFRICA AG AGAKHAN AGENCY AI AIG AIRBUS AIRFORCE AIRTEL AKDN AL ALIBABA ALIPAY ALLFINANZ ALLSTATE ALLY ALSACE ALSTOM AM AMAZON AMERICANEXPRESS AMERICANFAMILY AMEX AMFAM AMICA AMSTERDAM ANALYTICS ANDROID ANQUAN ANZ AO AOL APARTMENTS APP APPLE AQ AQUARELLE AR ARAB ARAMCO ARCHI ARMY ARPA ART ARTE AS ASDA ASIA ASSOCIATES AT ATHLETA ATTORNEY AU AUCTION AUDI AUDIBLE AUDIO AUSPOST AUTHOR AUTO AUTOS AW AWS AX AXA AZ AZURE BA BABY BAIDU BANAMEX BAND BANK BAR BARCELONA BARCLAYCARD BARCLAYS BAREFOOT BARGAINS BASEBALL BASKETBALL BAUHAUS BAYERN BB BBC BBT BBVA BCG BCN BD BE BEATS BEAUTY BEER BENTLEY BERLIN BEST BESTBUY BET BF BG BH BHARTI BI BIBLE BID BIKE BING BINGO BIO BIZ BJ BLACK BLACKFRIDAY BLOCKBUSTER BLOG BLOOMBERG BLUE BM BMS BMW BN BNPPARIBAS BO BOATS BOEHRINGER BOFA BOM BOND BOO BOOK BOOKING BOSCH BOSTIK BOSTON BOT BOUTIQUE BOX BR BRADESCO BRIDGESTONE BROADWAY BROKER BROTHER BRUSSELS BS BT BUILD BUILDERS BUSINESS BUY BUZZ BV BW BY BZ BZH CA CAB CAFE CAL CALL CALVINKLEIN CAM CAMERA CAMP CANON CAPETOWN CAPITAL CAPITALONE CAR CARAVAN CARDS CARE CAREER CAREERS CARS CASA CASE CASH CASINO CAT CATERING CATHOLIC CBA CBN CBRE CC CD CENTER CEO CERN CF CFA CFD CG CH CHANEL CHANNEL CHARITY CHASE CHAT CHEAP CHINTAI CHRISTMAS CHROME CHURCH CI CIPRIANI CIRCLE CISCO CITADEL CITI CITIC CITY CK CL CLAIMS CLEANING CLICK CLINIC CLINIQUE CLOTHING CLOUD CLUB CLUBMED CM CN CO COACH CODES COFFEE COLLEGE COLOGNE COM COMMBANK COMMUNITY COMPANY COMPARE COMPUTER COMSEC CONDOS CONSTRUCTION CONSULTING CONTACT CONTRACTORS COOKING COOL COOP CORSICA COUNTRY COUPON COUPONS COURSES CPA CR CREDIT CREDITCARD CREDITUNION CRICKET CROWN CRS CRUISE CRUISES CU CUISINELLA CV CW CX CY CYMRU CYOU CZ DABUR DAD DANCE DATA DATE DATING DATSUN DAY DCLK DDS DE DEAL DEALER DEALS DEGREE DELIVERY DELL DELOITTE DELTA DEMOCRAT DENTAL DENTIST DESI DESIGN DEV DHL DIAMONDS DIET DIGITAL DIRECT DIRECTORY DISCOUNT DISCOVER DISH DIY DJ DK DM DNP DO DOCS DOCTOR DOG DOMAINS DOT DOWNLOAD DRIVE DTV DUBAI DUNLOP DUPONT DURBAN DVAG DVR DZ EARTH EAT EC ECO EDEKA EDU EDUCATION EE EG EMAIL EMERCK ENERGY ENGINEER ENGINEERING ENTERPRISES EPSON EQUIPMENT ER ERICSSON ERNI ES ESQ ESTATE ET EU EUROVISION EUS EVENTS EXCHANGE EXPERT EXPOSED EXPRESS EXTRASPACE FAGE FAIL FAIRWINDS FAITH FAMILY FAN FANS FARM FARMERS FASHION FAST FEDEX FEEDBACK FERRARI FERRERO FI FIDELITY FIDO FILM FINAL FINANCE FINANCIAL FIRE FIRESTONE FIRMDALE FISH FISHING FIT FITNESS FJ FK FLICKR FLIGHTS FLIR FLORIST FLOWERS FLY FM FO FOO FOOD FOOTBALL FORD FOREX FORSALE FORUM FOUNDATION FOX FR FREE FRESENIUS FRL FROGANS FRONTIER FTR FUJITSU FUN FUND FURNITURE FUTBOL FYI GA GAL GALLERY GALLO GALLUP GAME GAMES GAP GARDEN GAY GB GBIZ GD GDN GE GEA GENT GENTING GEORGE GF GG GGEE GH GI GIFT GIFTS GIVES GIVING GL GLASS GLE GLOBAL GLOBO GM GMAIL GMBH GMO GMX GN GODADDY GOLD GOLDPOINT GOLF GOO GOODYEAR GOOG GOOGLE GOP GOT GOV GP GQ GR GRAINGER GRAPHICS GRATIS GREEN GRIPE GROCERY GROUP GS GT GU GUCCI GUGE GUIDE GUITARS GURU GW GY HAIR HAMBURG HANGOUT HAUS HBO HDFC HDFCBANK HEALTH HEALTHCARE HELP HELSINKI HERE HERMES HIPHOP HISAMITSU HITACHI HIV HK HKT HM HN HOCKEY HOLDINGS HOLIDAY HOMEDEPOT HOMEGOODS HOMES HOMESENSE HONDA HORSE HOSPITAL HOST HOSTING HOT HOTELS HOTMAIL HOUSE HOW HR HSBC HT HU HUGHES HYATT HYUNDAI IBM ICBC ICE ICU ID IE IEEE IFM IKANO IL IM IMAMAT IMDB IMMO IMMOBILIEN IN INC INDUSTRIES INFINITI INFO ING INK INSTITUTE INSURANCE INSURE INT INTERNATIONAL INTUIT INVESTMENTS IO IPIRANGA IQ IR IRISH IS ISMAILI IST ISTANBUL IT ITAU ITV JAGUAR JAVA JCB JE JEEP JETZT JEWELRY JIO JLL JM JMP JNJ JO JOBS JOBURG JOT JOY JP JPMORGAN JPRS JUEGOS JUNIPER KAUFEN KDDI KE KERRYHOTELS KERRYLOGISTICS KERRYPROPERTIES KFH KG KH KI KIA KIDS KIM KINDLE KITCHEN KIWI KM KN KOELN KOMATSU KOSHER KP KPMG KPN KR KRD KRED KUOKGROUP KW KY KYOTO KZ LA LACAIXA LAMBORGHINI LAMER LANCASTER LAND LANDROVER LANXESS LASALLE LAT LATINO LATROBE LAW LAWYER LB LC LDS LEASE LECLERC LEFRAK LEGAL LEGO LEXUS LGBT LI LIDL LIFE LIFEINSURANCE LIFESTYLE LIGHTING LIKE LILLY LIMITED LIMO LINCOLN LINK LIPSY LIVE LIVING LK LLC LLP LOAN LOANS LOCKER LOCUS LOL LONDON LOTTE LOTTO LOVE LPL LPLFINANCIAL LR LS LT LTD LTDA LU LUNDBECK LUXE LUXURY LV LY MA MADRID MAIF MAISON MAKEUP MAN MANAGEMENT MANGO MAP MARKET MARKETING MARKETS MARRIOTT MARSHALLS MATTEL MBA MC MCKINSEY MD ME MED MEDIA MEET MELBOURNE MEME MEMORIAL MEN MENU MERCKMSD MG MH MIAMI MICROSOFT MIL MINI MINT MIT MITSUBISHI MK ML MLB MLS MM MMA MN MO MOBI MOBILE MODA MOE MOI MOM MONASH MONEY MONSTER MORMON MORTGAGE MOSCOW MOTO MOTORCYCLES MOV MOVIE MP MQ MR MS MSD MT MTN MTR MU MUSEUM MUSIC MV MW MX MY MZ NA NAB NAGOYA NAME NATURA NAVY NBA NC NE NEC NET NETBANK NETFLIX NETWORK NEUSTAR NEW NEWS NEXT NEXTDIRECT NEXUS NF NFL NG NGO NHK NI NICO NIKE NIKON NINJA NISSAN NISSAY NL NO NOKIA NORTON NOW NOWRUZ NOWTV NP NR NRA NRW NTT NU NYC NZ OBI OBSERVER OFFICE OKINAWA OLAYAN OLAYANGROUP OLLO OM OMEGA ONE ONG ONL ONLINE OOO OPEN ORACLE ORANGE ORG ORGANIC ORIGINS OSAKA OTSUKA OTT OVH PA PAGE PANASONIC PARIS PARS PARTNERS PARTS PARTY PAY PCCW PE PET PF PFIZER PG PH PHARMACY PHD PHILIPS PHONE PHOTO PHOTOGRAPHY PHOTOS PHYSIO PICS PICTET PICTURES PID PIN PING PINK PIONEER PIZZA PK PL PLACE PLAY PLAYSTATION PLUMBING PLUS PM PN PNC POHL POKER POLITIE PORN POST PR PRAMERICA PRAXI PRESS PRIME PRO PROD PRODUCTIONS PROF PROGRESSIVE PROMO PROPERTIES PROPERTY PROTECTION PRU PRUDENTIAL PS PT PUB PW PWC PY QA QPON QUEBEC QUEST RACING RADIO RE READ REALESTATE REALTOR REALTY RECIPES RED REDSTONE REDUMBRELLA REHAB REISE REISEN REIT RELIANCE REN RENT RENTALS REPAIR REPORT REPUBLICAN REST RESTAURANT REVIEW REVIEWS REXROTH RICH RICHARDLI RICOH RIL RIO RIP RO ROCKS RODEO ROGERS ROOM RS RSVP RU RUGBY RUHR RUN RW RWE RYUKYU SA SAARLAND SAFE SAFETY SAKURA SALE SALON SAMSCLUB SAMSUNG SANDVIK SANDVIKCOROMANT SANOFI SAP SARL SAS SAVE SAXO SB SBI SBS SC SCB SCHAEFFLER SCHMIDT SCHOLARSHIPS SCHOOL SCHULE SCHWARZ SCIENCE SCOT SD SE SEARCH SEAT SECURE SECURITY SEEK SELECT SENER SERVICES SEVEN SEW SEX SEXY SFR SG SH SHANGRILA SHARP SHAW SHELL SHIA SHIKSHA SHOES SHOP SHOPPING SHOUJI SHOW SI SILK SINA SINGLES SITE SJ SK SKI SKIN SKY SKYPE SL SLING SM SMART SMILE SN SNCF SO SOCCER SOCIAL SOFTBANK SOFTWARE SOHU SOLAR SOLUTIONS SONG SONY SOY SPA SPACE SPORT SPOT SR SRL SS ST STADA STAPLES STAR STATEBANK STATEFARM STC STCGROUP STOCKHOLM STORAGE STORE STREAM STUDIO STUDY STYLE SU SUCKS SUPPLIES SUPPLY SUPPORT SURF SURGERY SUZUKI SV SWATCH SWISS SX SY SYDNEY SYSTEMS SZ TAB TAIPEI TALK TAOBAO TARGET TATAMOTORS TATAR TATTOO TAX TAXI TC TCI TD TDK TEAM TECH TECHNOLOGY TEL TEMASEK TENNIS TEVA TF TG TH THD THEATER THEATRE TIAA TICKETS TIENDA TIPS TIRES TIROL TJ TJMAXX TJX TK TKMAXX TL TM TMALL TN TO TODAY TOKYO TOOLS TOP TORAY TOSHIBA TOTAL TOURS TOWN TOYOTA TOYS TR TRADE TRADING TRAINING TRAVEL TRAVELERS TRAVELERSINSURANCE TRUST TRV TT TUBE TUI TUNES TUSHU TV TVS TW TZ UA UBANK UBS UG UK UNICOM UNIVERSITY UNO UOL UPS US UY UZ VA VACATIONS VANA VANGUARD VC VE VEGAS VENTURES VERISIGN VERSICHERUNG VET VG VI VIAJES VIDEO VIG VIKING VILLAS VIN VIP VIRGIN VISA VISION VIVA VIVO VLAANDEREN VN VODKA VOLVO VOTE VOTING VOTO VOYAGE VU WALES WALMART WALTER WANG WANGGOU WATCH WATCHES WEATHER WEATHERCHANNEL WEBCAM WEBER WEBSITE WED WEDDING WEIBO WEIR WF WHOSWHO WIEN WIKI WILLIAMHILL WIN WINDOWS WINE WINNERS WME WOLTERSKLUWER WOODSIDE WORK WORKS WORLD WOW WS WTC WTF XBOX XEROX XIHUAN XIN XN--11B4C3D XN--1CK2E1B XN--1QQW23A XN--2SCRJ9C XN--30RR7Y XN--3BST00M XN--3DS443G XN--3E0B707E XN--3HCRJ9C XN--3PXU8K XN--42C2D9A XN--45BR5CYL XN--45BRJ9C XN--45Q11C XN--4DBRK0CE XN--4GBRIM XN--54B7FTA0CC XN--55QW42G XN--55QX5D XN--5SU34J936BGSG XN--5TZM5G XN--6FRZ82G XN--6QQ986B3XL XN--80ADXHKS XN--80AO21A XN--80AQECDR1A XN--80ASEHDB XN--80ASWG XN--8Y0A063A XN--90A3AC XN--90AE XN--90AIS XN--9DBQ2A XN--9ET52U XN--9KRT00A XN--B4W605FERD XN--BCK1B9A5DRE4C XN--C1AVG XN--C2BR7G XN--CCK2B3B XN--CCKWCXETD XN--CG4BKI XN--CLCHC0EA0B2G2A9GCD XN--CZR694B XN--CZRS0T XN--CZRU2D XN--D1ACJ3B XN--D1ALF XN--E1A4C XN--ECKVDTC9D XN--EFVY88H XN--FCT429K XN--FHBEI XN--FIQ228C5HS XN--FIQ64B XN--FIQS8S XN--FIQZ9S XN--FJQ720A XN--FLW351E XN--FPCRJ9C3D XN--FZC2C9E2C XN--FZYS8D69UVGM XN--G2XX48C XN--GCKR3F0F XN--GECRJ9C XN--GK3AT1E XN--H2BREG3EVE XN--H2BRJ9C XN--H2BRJ9C8C XN--HXT814E XN--I1B6B1A6A2E XN--IMR513N XN--IO0A7I XN--J1AEF XN--J1AMH XN--J6W193G XN--JLQ480N2RG XN--JVR189M XN--KCRX77D1X4A XN--KPRW13D XN--KPRY57D XN--KPUT3I XN--L1ACC XN--LGBBAT1AD8J XN--MGB9AWBF XN--MGBA3A3EJT XN--MGBA3A4F16A XN--MGBA7C0BBN0A XN--MGBAAM7A8H XN--MGBAB2BD XN--MGBAH1A3HJKRD XN--MGBAI9AZGQP6J XN--MGBAYH7GPA XN--MGBBH1A XN--MGBBH1A71E XN--MGBC0A9AZCG XN--MGBCA7DZDO XN--MGBCPQ6GPA1A XN--MGBERP4A5D4AR XN--MGBGU82A XN--MGBI4ECEXP XN--MGBPL2FH XN--MGBT3DHD XN--MGBTX2B XN--MGBX4CD0AB XN--MIX891F XN--MK1BU44C XN--MXTQ1M XN--NGBC5AZD XN--NGBE9E0A XN--NGBRX XN--NODE XN--NQV7F XN--NQV7FS00EMA XN--NYQY26A XN--O3CW4H XN--OGBPF8FL XN--OTU796D XN--P1ACF XN--P1AI XN--PGBS0DH XN--PSSY2U XN--Q7CE6A XN--Q9JYB4C XN--QCKA1PMC XN--QXA6A XN--QXAM XN--RHQV96G XN--ROVU88B XN--RVC1E0AM3E XN--S9BRJ9C XN--SES554G XN--T60B56A XN--TCKWE XN--TIQ49XQYJ XN--UNUP4Y XN--VERMGENSBERATER-CTB XN--VERMGENSBERATUNG-PWB XN--VHQUV XN--VUQ861B XN--W4R85EL8FHU5DNRA XN--W4RS40L XN--WGBH1C XN--WGBL6A XN--XHQ521B XN--XKC2AL3HYE2A XN--XKC2DL3A5EE0H XN--Y9A3AQ XN--YFRO4I67O XN--YGBI2AMMX XN--ZFR164B XXX XYZ YACHTS YAHOO YAMAXUN YANDEX YE YODOBASHI YOGA YOKOHAMA YOU YOUTUBE YT YUN ZA ZAPPOS ZARA ZERO ZIP ZM ZONE ZUERICH ZW firecat53-urlscan-03f47eb/urlscan/urlchoose.py000066400000000000000000001134161464200272300214120ustar00rootroot00000000000000# Copyright (C) 2006-2007 Daniel Burrows # Copyright (C) 2023 Scott Hansen # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA """An urwid listview-based widget that lets you choose a URL from a list of URLs.""" import contextlib import json import os from os.path import dirname, exists, expanduser from sys import platform import re import shlex import subprocess import sys from threading import Thread import webbrowser import urwid import urwid.curses_display import urwid.raw_display if platform == 'darwin': COPY_COMMANDS = ('pbcopy',) COPY_COMMANDS_PRIMARY = ('pbcopy',) elif 'WAYLAND_DISPLAY' in os.environ: COPY_COMMANDS = ('wl-copy',) COPY_COMMANDS_PRIMARY = ('wl-copy --primary',) else: COPY_COMMANDS = ("xsel -ib", "xclip -i -selection clipboard") COPY_COMMANDS_PRIMARY = ("xsel -i", "xclip -i") def shorten_url(url, cols, shorten): """Shorten long URLs to fit on one line. """ cols = ((cols - 6) * .85) # 6 cols for urlref and don't use while line if shorten is False or len(url) < cols: return url split = int(cols * .5) return url[:split] + "..." + url[-split:] def grp_list(items): """Organize list of items [a,2,3,4,a,4,2,a,1, etc...] like: [[a,2,3,4], [a,4,2], [a,1]], where 'a' is a urwid.Divider """ grp = [] res = [] for item in items: if isinstance(item, urwid.Divider): res.append(grp) grp = [items[0]] else: grp.append(item) res.append(grp) return res[1:] def splittext(text, search, attr): """Split a text string by search string and add Urwid display attribute to the search term. Args: text - string search - search string attr - attribute string to add Returns: urwid markup list ["string", ("default", " mo"), "re string"] for search="mo", text="string more string" and attr="default" """ if search: pat = re.compile(f"({re.escape(search)})", re.IGNORECASE) else: return text final = pat.split(text) final = [(attr, i) if i.lower() == search.lower() else i for i in final] return final class URLChooser: def __init__(self, extractedurls, compact=False, reverse=False, nohelp=False, dedupe=False, shorten=True, run="", runsafe="", single=False, pipe=False, genconf=False, width=0, whitespaceoff=False): self.conf = expanduser("~/.config/urlscan/config.json") self.keys = {'/': self._search_key, '0': self._digits, '1': self._digits, '2': self._digits, '3': self._digits, '4': self._digits, '5': self._digits, '6': self._digits, '7': self._digits, '8': self._digits, '9': self._digits, 'a': self._add_url, 'C': self._clipboard, 'c': self._context, 'ctrl l': self._clear_screen, 'd': self._del_url, 'f1': self._help_menu, 'G': self._bottom, 'g': self._top, 'j': self._down, 'k': self._up, 'J': self._next, 'K': self._previous, 'P': self._clipboard_pri, 'l': self._link_handler, 'o': self._open_queue, 'O': self._open_queue_win, 'p': self._palette, 'Q': self._quit, 'q': self._quit, 'R': self._reverse, 'S': self._all_shorten, 's': self._shorten, 'u': self._all_escape } self.palettes = {} # Default color palette default = [('header', 'white', 'dark blue', 'standout'), ('footer', 'white', 'dark red', 'standout'), ('search', 'white', 'dark green', 'standout'), ('msgtext', '', ''), ('msgtext:ellipses', 'light gray', 'black'), ('urlref:number:braces', 'light gray', 'black'), ('urlref:number', 'yellow', 'black', 'standout'), ('urlref:url', 'white', 'black', 'standout'), ('url:sel', 'white', 'dark blue', 'bold')] # Default black & white palette blw = [('header', 'black', 'light gray', 'standout'), ('footer', 'black', 'light gray', 'standout'), ('search', 'black', 'light gray', 'standout'), ('msgtext', '', ''), ('msgtext:ellipses', 'white', 'black'), ('urlref:number:braces', 'white', 'black'), ('urlref:number', 'white', 'black', 'standout'), ('urlref:url', 'white', 'black', 'standout'), ('url:sel', 'black', 'light gray', 'bold')] # Boruch's colorized palette colorized = [('header', 'brown', 'black', 'standout'), ('footer', 'white', 'dark red', 'standout'), ('search', 'white', 'dark green', 'standout'), ('msgtext', 'light cyan', 'black'), ('msgtext:ellipses', 'light gray', 'black'), ('urlref:number:braces', 'light gray', 'black'), ('urlref:number', 'yellow', 'black', 'standout'), ('urlref:url', 'dark green', 'black', 'standout'), ('url:sel', 'white', 'black', '')] self.palettes.update([("default", default), ("bw", blw), ("colorized", colorized)]) if genconf is True: self._config_create() try: with open(self.conf, 'r', encoding=sys.getdefaultencoding()) as conf_file: data = json.load(conf_file) try: for pal_name, pal in data['palettes'].items(): self.palettes.update([(pal_name, [tuple(i) for i in pal])]) except KeyError: pass try: items = data['keys'].items() for key, value in items: if value: if value == "open_url": urwid.Button._command_map._command[key] = 'activate' value = getattr(self, f"_{value}") else: del self.keys[key] continue self.keys.update([(key, value)]) except KeyError: pass except FileNotFoundError: pass try: subprocess.run(['xdg-open'], check=False, stdout=subprocess.DEVNULL) self.xdg = True except OSError: self.xdg = False self.shorten = shorten self.compact = compact self.queue = [] self.run = run self.runsafe = runsafe self.single = single self.pipe = pipe self.search = False self.search_string = "" self.no_matches = False self.enter = False self.term_width, _ = urwid.raw_display.Screen().get_cols_rows() self.width = min(self.term_width, width or self.term_width) self.whitespaceoff = whitespaceoff self.activate_keys = [i for i, j in urwid.Button._command_map._command.items() if j == 'activate'] self.items, self.urls = self.process_urls(extractedurls, dedupe=dedupe, shorten=self.shorten) # Original version of all items self.items_orig = self.items # Store items grouped into sections self.items_org = grp_list(self.items) # Store 'compact' mode items self.items_com = [i for i in self.items if isinstance(i, urwid.Columns) is True] if self.compact is True: self.items, self.items_com = self.items_com, self.items self.urls_unesc = [i.replace('\\', '') for i in self.urls] self.unesc = False listbox = urwid.ListBox(self.items) self.header = (":: F1 - help/keybindings :: " "q - quit :: " "/ - search :: " "URL opening mode - {} :: " "Queue - {}") self.link_open_modes = ["Web Browser", "Xdg-Open"] if self.xdg is True else ["Web Browser"] if self.runsafe: self.link_open_modes.insert(0, self.runsafe) elif self.run: self.link_open_modes.insert(0, self.run) self.nohelp = nohelp if nohelp is False: self.headerwid = urwid.AttrMap(urwid.Text( self.header.format(self.link_open_modes[0], len(self.queue))), 'header') else: self.headerwid = None self.top = urwid.Frame(listbox, self.headerwid) self.pad = self.term_width - self.width self.top = urwid.Padding(self.top, left=0, right=self.pad) if self.urls: self.top.base_widget.body.focus_position = \ (2 if self.compact is False else 0) if reverse is True: self._reverse() self.tui = urwid.curses_display.Screen() self.palette_names = list(self.palettes.keys()) self.palette_idx = 0 self.number = "" self.help_menu = False def main(self): """Urwid main event loop """ self.loop = urwid.MainLoop(self.top, self.palettes[self.palette_names[0]], screen=self.tui, handle_mouse=False, input_filter=self.handle_keys, unhandled_input=self.unhandled) self.loop.run() @property def size(self): _, rows = self.tui.get_cols_rows() return (self.width, rows) def handle_keys(self, keys, raw): """Handle widget default keys - 'Enter' or 'space' to load URL - 'Enter' to end search mode - add 'space' to search string in search mode - Workaround some small positioning bugs """ for j, k in enumerate(keys): if self.search is True: text = f"Search: {self.search_string}" if k == 'enter': # Catch 'enter' key to prevent opening URL in mkbrowseto self.enter = True if not self.items: self.search = False self.enter = False if self.search_string: footer = 'search' else: footer = 'default' text = "" footerwid = urwid.AttrMap(urwid.Text(text), footer) self.top.base_widget.footer = footerwid elif k in self.activate_keys: self.search_string += k self._search() elif k == 'backspace': self.search_string = self.search_string[:-1] self._search() elif k in self.activate_keys and \ self.urls and \ self.search is False and \ self.help_menu is False: self._open_url() elif self.help_menu is True: self._help_menu() return [] if k == 'up': # Works around bug where the up arrow goes higher than the top list # item and unintentionally triggers context and palette switches. # Remaps 'up' to 'k' keys[j] = 'k' if k == 'home': # Remap 'home' to 'g'. Works around small bug where 'home' takes the cursor # above the top list item. keys[j] = 'g' # filter backspace out before the widget, it has a weird interaction return [i for i in keys if i != 'backspace'] def unhandled(self, key): """Handle other keyboard actions not handled by the ListBox widget. """ self.key = key if self.search is True: if self.enter is False and self.no_matches is False: if len(key) == 1 and key.isprintable(): self.search_string += key self._search() elif self.enter is True and not self.search_string: self.search = False self.enter = False return if not self.urls and key not in "Qq": return # No other actions are useful with no URLs if self.help_menu is False: try: self.keys[key]() except KeyError: pass def _quit(self): """q/Q""" raise urwid.ExitMainLoop() def _open_url(self): """ or """ load_text = "Loading URL..." if self.link_open_modes[0] != (self.run or self.runsafe) \ else f"Executing: {self.run or self.runsafe}" if os.environ.get('BROWSER') not in ['elinks', 'links', 'w3m', 'lynx']: self._footer_display(load_text, 5) def _background_queue(self, mode): """Open URLs in background""" for url in self.queue: self.mkbrowseto(url, mode=mode)() self.draw_screen() def _queue(self, mode=2): """Open all URLs in queue Args: mode - 2 for new tab, 1 for new window """ load_text = "Loading URLs in queue..." \ if self.link_open_modes[0] != (self.run or self.runsafe) \ else f"Executing: {self.run or self.runsafe}" if os.environ.get('BROWSER') in ['elinks', 'links', 'w3m', 'lynx']: self._footer_display("Opening multiple links not support in text browsers", 5) else: self._footer_display(load_text, 5) thr = Thread(target=self._background_queue, args=(mode,)) thr.start() self.queue = [] self.headerwid = urwid.AttrMap(urwid.Text( self.header.format(self.link_open_modes[0], len(self.queue))), 'header') self.top.base_widget.header = self.headerwid def _open_queue(self): """o (new tab)""" if self.queue: self._queue() def _open_queue_win(self): """O (new window)""" if self.queue: self._queue(1) def _add_url(self): """a""" fpo = self.top.base_widget.body.focus_position url_idx = len([i for i in self.items[:fpo + 1] if isinstance(i, urwid.Columns)]) - 1 if self.compact is False and fpo <= 1: return self.queue.append(self.urls[url_idx]) self.queue = list(set(self.queue)) self.headerwid = urwid.AttrMap(urwid.Text( self.header.format(self.link_open_modes[0], len(self.queue))), 'header') self.top.base_widget.header = self.headerwid label = self.items[fpo][1].label if not label.startswith("* "): self.items[fpo][1].set_label(f"* {label}") def _del_url(self): """d""" fpo = self.top.base_widget.body.focus_position url_idx = len([i for i in self.items[:fpo + 1] if isinstance(i, urwid.Columns)]) - 1 if self.compact is False and fpo <= 1: return try: self.queue.remove(self.urls[url_idx]) self.headerwid = urwid.AttrMap(urwid.Text( self.header.format(self.link_open_modes[0], len(self.queue))), 'header') self.top.base_widget.header = self.headerwid label = self.items[fpo][1].label if label.startswith("* "): self.items[fpo][1].set_label(label.lstrip("* ")) except ValueError: pass def _help_menu(self): """F1""" if self.help_menu is False: self.focus_pos_saved = self.top.base_widget.body.focus_position help_men = "\n".join([f"{i} - {j.__name__.strip('_')}" for i, j in self.keys.items() if j.__name__ != '_digits']) help_men = "KEYBINDINGS\n" + help_men + "\n<0-9> - Jump to item" docs = ("OPTIONS\n" "add_url -- add URL to queue\n" "all_escape -- toggle unescape all URLs\n" "all_shorten -- toggle shorten all URLs\n" "bottom -- move cursor to last item\n" "clear_screen -- redraw screen\n" "clipboard -- copy highlighted URL to clipboard\n" " using xsel/xclip\n" "clipboard_pri -- copy highlighted URL to primary\n" " selection using xsel/xclip\n" "config_create -- create ~/.config/urlscan/config.json\n" "context -- show/hide context\n" "del_url -- delete URL from queue\n" "down -- cursor down\n" "help_menu -- show/hide help menu\n" "link_handler -- cycle through xdg-open, webbrowser \n" " and user-defined function\n" "next -- jump to next URL\n" "open_queue -- open all URLs in queue\n" "open_queue_win-- open all URLs in queue in new window\n" "open_url -- open selected URL\n" "palette -- cycle through palettes\n" "previous -- jump to previous URL\n" "quit -- quit\n" "reverse -- reverse order URLs/context\n" "shorten -- toggle shorten highlighted URL\n" "single -- quit urlscan after opening a\n" " single link\n" "top -- move to first list item\n" "up -- cursor up\n") self.top.base_widget.body = \ urwid.ListBox(urwid.SimpleListWalker([urwid.Columns([(24, urwid.Text(help_men)), urwid.Text(docs)])])) else: self.top.base_widget.body = urwid.ListBox(self.items) self.top.base_widget.body.focus_position = self.focus_pos_saved self.help_menu = not self.help_menu def _search_key(self): """ / """ if self.urls: self.search = True if self.compact is True: self._context() else: return self.no_matches = False self.search_string = "" # Reset the search highlighting self._search() footerwid = urwid.AttrMap(urwid.Text("Search: "), 'footer') self.top.base_widget.footer = footerwid self.items = self.items_orig self.top.base_widget.body = urwid.ListBox(self.items) def _digits(self): """ 0-9 """ self.number += self.key try: if self.compact is False: self.top.base_widget.body.focus_position = \ self.items.index(self.items_com[max(int(self.number) - 1, 0)]) else: self.top.base_widget.body.focus_position = \ self.items.index(self.items[max(int(self.number) - 1, 0)]) except IndexError: self.number = self.number[:-1] except ValueError: pass self.top.base_widget.keypress(self.size, "") # Trick urwid into redisplaying the cursor if self.number: self._footer_display(f"Selection: {self.number}", 1) def _clear_screen(self): """ Ctrl-l """ self.draw_screen() def _down(self): """ j """ self.top.base_widget.keypress(self.size, "down") def _up(self): """ k """ self.top.base_widget.keypress(self.size, "up") def _top(self): """ g """ # Goto top of the list self.top.base_widget.body.focus_position = 2 if self.compact is False else 0 self.top.base_widget.keypress(self.size, "") # Trick urwid into redisplaying the cursor def _bottom(self): """ G """ # Goto bottom of the list self.top.base_widget.body.focus_position = len(self.items) - 1 self.top.base_widget.keypress(self.size, "") # Trick urwid into redisplaying the cursor def _selectable_positions(self): return [i for i, item in enumerate(self.items) if item.selectable()] def _next(self): """ J """ current_position = self.top.base_widget.body.focus_position if current_position >= self._selectable_positions()[-1]: # Do not jump if focus is on or after the last selectable position return # Jump to the first selectable position after the currently focused position target_position = min(p for p in self._selectable_positions() if p > current_position) self.top.base_widget.body.focus_position = target_position self.top.base_widget.keypress(self.size, "") # Trick urwid into redisplaying the cursor def _previous(self): """ K """ current_position = self.top.base_widget.body.focus_position if current_position <= self._selectable_positions()[0]: # Do not jump if focus is on or before the first selectable position return # Jump to the first selectable position before the currently focused position target_position = max(p for p in self._selectable_positions() if p < current_position) self.top.base_widget.body.focus_position = target_position self.top.base_widget.keypress(self.size, "") # Trick urwid into redisplaying the cursor def _shorten(self): """ s """ # Toggle shortened URL for selected item fpo = self.top.base_widget.body.focus_position url_idx = len([i for i in self.items[:fpo + 1] if isinstance(i, urwid.Columns)]) - 1 if self.compact is False and fpo <= 1: return url = self.urls[url_idx] short = not "..." in self.items[fpo][1].label self.items[fpo][1].set_label(shorten_url(url, self.size[0], short)) def _all_shorten(self): """ S """ # Toggle all shortened URLs self.shorten = not self.shorten urls = iter(self.urls) for item in self.items: # Each Column has (Text, Button). Update the Button label if isinstance(item, urwid.Columns): item[1].set_label(shorten_url(next(urls), self.size[0], self.shorten)) def _all_escape(self): """ u """ # Toggle all escaped URLs self.unesc = not self.unesc self.urls, self.urls_unesc = self.urls_unesc, self.urls urls = iter(self.urls) for item in self.items: # Each Column has (Text, Button). Update the Button label if isinstance(item, urwid.Columns): item[1].set_label(shorten_url(next(urls), self.size[0], self.shorten)) def _reverse(self): """ R """ # Reverse items fpo = self.top.base_widget.body.focus_position if self.compact is True: self.items.reverse() else: rev = [] for item in self.items: if isinstance(item, urwid.Divider): rev.insert(0, item) elif isinstance(item, urwid.Text): rev.insert(1, item) else: rev.insert(2, item) self.items = rev self.top.base_widget.body = urwid.ListBox(self.items) self.top.base_widget.body.focus_position = self._cur_focus(fpo) def _context(self): """ c """ # Show/hide context if self.search_string: # Reset search when toggling compact mode footerwid = urwid.AttrMap(urwid.Text(""), 'default') self.top.base_widget.footer = footerwid self.search_string = "" self.items = self.items_orig fpo = self.top.base_widget.body.focus_position self.items, self.items_com = self.items_com, self.items self.top.base_widget.body = urwid.ListBox(self.items) self.top.base_widget.body.focus_position = self._cur_focus(fpo) self.compact = not self.compact def _clipboard(self, pri=False): """ C """ # Copy highlighted url to clipboard fpo = self.top.base_widget.body.focus_position url_idx = len([i for i in self.items[:fpo + 1] if isinstance(i, urwid.Columns)]) - 1 if self.compact is False and fpo <= 1: return url = self.urls[url_idx] cmds = COPY_COMMANDS_PRIMARY if pri else COPY_COMMANDS for cmd in cmds: try: subprocess.run(shlex.split(cmd), check=False, input=url.encode(sys.getdefaultencoding()), stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) self._footer_display("Copied url to " f"{'primary' if pri is True else 'clipboard'} selection", 5) except OSError: continue if self.single is True: self._quit() break def _clipboard_pri(self): """ P """ # Copy highlighted url to primary selection self._clipboard(pri=True) def _palette(self): """ p """ # Loop through available palettes self.palette_idx += 1 try: self.loop.screen.register_palette(self.palettes[self.palette_names[self.palette_idx]]) except IndexError: self.loop.screen.register_palette(self.palettes[self.palette_names[0]]) self.palette_idx = 0 self.loop.screen.clear() def _config_create(self): """ --genconf """ # Create ~/.config/urlscan/config.json if if doesn't exist if not exists(self.conf): os.makedirs(dirname(expanduser(self.conf)), exist_ok=True) keys = dict(zip(self.keys.keys(), [i.__name__.strip('_') for i in self.keys.values()])) with open(expanduser(self.conf), 'w', encoding=sys.getdefaultencoding()) as pals: pals.writelines(json.dumps({"palettes": self.palettes, "keys": keys}, indent=4)) print("Created ~/.config/urlscan/config.json") else: print("~/.config/urlscan/config.json already exists") def _footer_display(self, text, time): """Display given text in the footer. Clears after