bleachbit-4.4.2/0000775000175000017500000000000014144024254012362 5ustar fabiofabiobleachbit-4.4.2/debian/0000775000175000017500000000000014144024254013604 5ustar fabiofabiobleachbit-4.4.2/org.bleachbit.BleachBit.metainfo.xml0000664000175000017500000000350714144024253021230 0ustar fabiofabio org.bleachbit.BleachBit CC0-1.0 GPL-3.0 BleachBit org.bleachbit.BleachBit.desktop Cleans files to free disk space and to maintain privacy

BleachBit quickly frees disk space and tirelessly guards your privacy. Free cache, delete cookies, clear Internet history, shred temporary files, delete logs, and discard junk you didn't know was there. It wipes clean a thousand applications including Firefox, Adobe Flash, Google Chrome, Opera, and more.

Beyond simply deleting files, BleachBit includes advanced features such as shredding files to prevent recovery, wiping free disk space to hide traces of files deleted by other applications, and vacuuming Firefox to make it faster.

Default dark mode, showing found junk files https://www.bleachbit.org/images/bleachbit320_windows10_preview.png https://www.bleachbit.org/ https://github.com/bleachbit/bleachbit/issues https://docs.bleachbit.org/ https://www.bleachbit.org/contribute https://www.bleachbit.org/contribute/translate andrew@bleachbit.org bleachbit bleachbit-4.4.2/org.bleachbit.BleachBit.desktop0000664000175000017500000001621514144024253020300 0ustar fabiofabio[Desktop Entry] Version=1.1 Type=Application Name=BleachBit Comment=Free space and maintain privacy Comment[af]=Maak ruimte en behou privaatheid Comment[ar]=لإخلاء مساحات التخزين و الحفاظ على الخصوصية Comment[ast]=Llibera espaciu y caltién la privacidá Comment[be]=Вольная прастора і кіраваньне сакрэтнасьцю Comment[bg]=Освободете място и запазете поверителност Comment[bn]=স্পেস খালি করুন এবং গোপনীয়তা রক্ষা করুন Comment[bs]=Slobodan prostor i čuvanje privatnosti Comment[ca]=Allibereu espai i mantingueu la privacitat Comment[cs]=Uvolní místo na disku a ochrání vaše soukromí Comment[da]=Frigør plads og beskyt privatlivet Comment[de]=Speicherplatz freigeben und die Privatsphäre schützen Comment[el]=Ελευθερώστε χώρο και διατηρήστε το ιδιωτικό σας απόρρητο Comment[eo]=Liberigi lokon kaj prizorgi privatecon Comment[es]=Libere espacio y preserve su privacidad Comment[et]=Vabasta ruumi ja säilita privaatsus Comment[eu]=Egin leku librea eta mantendu pribatutasuna Comment[fa]=ایجاد فضای آزاد و برقراری امنیت Comment[fi]=Vapauta tilaa ja hallitse yksityisyyttä Comment[fr]=Libérez de l'espace et préservez votre vie privée Comment[gl]=Liberar espazo e manter a intimidade Comment[he]=פינוי מקום ושמירה על הפרטיות Comment[hi]=जगह खाली करें और गोपनीयता मेनटेन रखें Comment[hr]=Oslobodite prostor i očuvajte privatnost Comment[hu]=Hely felszabadítása és magánszférájának fenntartása Comment[ia]=Liberar spatio e mantener le confidentialitate Comment[id]=Bersihkan ruang dan perbaiki privasi Comment[ie]=Liberar spacie e mantener confidentie Comment[it]=Libera spazio e mantiene la privacy Comment[ja]=スペースの確保とプライバシーの保全 Comment[ko]=개인정보 유지 및 공간 확보 Comment[ku]=Ciyê vala û parastina nepeniyê Comment[lt]=Atlaisvinkite vietą bei apsaugokite privatumą pašalindami nereikalingus failus ir duomenis Comment[ms]=Kosongkan ruang dan kekalkan kerahsiaan Comment[nb]=Frigjør plass og beskytt personvernet Comment[nl]=Maak ruimte vrij en behoud uw privacy Comment[nn]=Frigjer plass og oppretthald personvern Comment[pl]=Zwolnij miejsce na dysku i zachowaj prywatność Comment[pt]=Libertar espaço e manter privacidade Comment[pt_BR]=Libera espaço e mantém sua privacidade Comment[ro]=Spațiu liber și păstrarea intimității Comment[ru]=Освобождение дискового пространства и обслуживание конфиденциальности Comment[sk]=Voľné miesto a zachovanie súkromia Comment[sl]=Sprostite prostor in ohranite zasebnost Comment[sq]=Liro hapësirë dhe mbaj privacinë Comment[sr]=Ослобађање простора и чување приватности Comment[sv]=Frigör utrymme och bibehåll integritet Comment[ta]=இடத்தை வெற்றாக்கவும் தகவல் பாதுகாப்பை பேணவும் Comment[te]=స్థలాన్ని ఖాళీ చేసి, గోప్యతను పాటించండి Comment[th]=พื้นที่ว่างและการรักษาความเป็นส่วนตัว Comment[tr]=Alanı boşalt ve gizliliği sağla Comment[uk]=Вільний простір та керування приватністю Comment[uz]=Bo‘sh joy va asosiy maxfiylik Comment[vi]=Làm trống ổ đĩa và bảo vệ sự riêng tư Comment[yi]=פריי פּלאַץ און טייַנען פּריוואַטקייט Comment[zh_CN]=释放空间并保护隐私 Comment[zh_TW]=釋放空間並維護隱私 GenericName=Unnecessary file cleaner GenericName[af]=Onnodige lêer skoonmaker GenericName[ar]=منظّف الملفات غير الضرورية GenericName[ast]=Llimpiador de ficheros innecesarios GenericName[be]=Неабавязковы уборшчык файлаў GenericName[bg]=Изчистване на непотребни файлове GenericName[bn]=অপ্রয়োজনীয় ফাইল ক্লিনার GenericName[bs]=Nepotrebno čišćenje datoteka GenericName[ca]=Netejador de fitxers innecessaris GenericName[cs]=Čistič disku od nepotřebných souborů GenericName[da]=Unødvendig fil-oprydder GenericName[de]=Reiniger für unnötige Dateien GenericName[el]=Καθαριστής περιττών αρχείων GenericName[eo]=Purigilo de nenecesaj dosieroj GenericName[es]=Limpiador de archivos innecesarios GenericName[et]=Mittevajalike failide puhastaja GenericName[eu]=Beharrezko ez diren fitxategien garbitzailea GenericName[fa]=پاک کننده پرونده های بدرد نخور GenericName[fi]=Turhien tiedostojen puhdistaja GenericName[fr]=Nettoyeur de fichiers inutiles GenericName[gl]=Limpador de ficheiros innecesarios GenericName[he]=מנקה הקבצים הבלתי נחוצים GenericName[hi]=अनावश्यक फ़ाइल क्लीनर GenericName[hr]=Čistač nepotrebnih datoteka GenericName[hu]=Szükségtelen fájl törlő GenericName[ia]=Nettator de files non necessari GenericName[id]=Pembersih berkas yang tidak diperlukan GenericName[ie]=Nettator de ínnecessi files GenericName[it]=Ripulisce file non necessari GenericName[ja]=不要ファイルクリーナー GenericName[ko]=불필요한 파일 클리너 GenericName[ku]=Paqijkirina pelên nehewce GenericName[ky]=Керексиз файлдардын тазалагычы GenericName[lt]=Nereikalingų failų šalinimas GenericName[ms]=Pembersih fail yang tidak diperlukan GenericName[nb]=Unødvendig filrenser GenericName[nl]=Verwijder overbodige bestanden GenericName[nn]=Renskar for unødige filer GenericName[pl]=Program czyszczący niepotrzebne pliki GenericName[pt]=Limpeza de ficheiros desnecessários GenericName[pt_BR]=Limpar arquivos desnecessários GenericName[ro]=Curățarea fișierelor inutile GenericName[ru]=Очистка от ненужных файлов GenericName[sk]=Čistenie nepotrebných súborov GenericName[sl]=Čistilnik nepotrebnih datotek GenericName[sq]=Pastrues filesh i panevojshëm GenericName[sr]=Чистач непотребних датотека GenericName[sv]=Städare av onödiga filer GenericName[ta]=தேவையற்ற கோப்புகளை நீக்கும் மென்பொருள் GenericName[te]=అనవసరమైన ఫైల్ క్లీనర్ GenericName[th]=เครื่องมือทำความสะอาดแฟ้มที่ไม่จำเป็น GenericName[tr]=Gereksiz dosya temizleyici GenericName[uk]=Необов'язкова чистка файлів GenericName[uz]=Keraksiz fayl tozalagich GenericName[vi]=Trình dọn dẹp tập tin không cần thiết GenericName[yi]=ומנייטיק טעקע קלינער GenericName[zh_CN]=不必要文件清理器 GenericName[zh_TW]=無用檔案清理器 TryExec=bleachbit Exec=bleachbit Icon=bleachbit Categories=System;FileTools;GTK; Keywords=cache;clean;free;performance;privacy; StartupNotify=true X-GNOME-UsesNotifications=true bleachbit-4.4.2/bleachbit/0000775000175000017500000000000014144024254014277 5ustar fabiofabiobleachbit-4.4.2/bleachbit/GUI.py0000775000175000017500000014106114144024253015302 0ustar fabiofabio#!/usr/bin/python3 # vim: ts=4:sw=4:expandtab # BleachBit # Copyright (C) 2008-2021 Andrew Ziem # https://www.bleachbit.org # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . """ GTK graphical user interface """ from bleachbit import GuiBasic from bleachbit import Cleaner, FileUtilities from bleachbit import _, APP_NAME, appicon_path, portable_mode, windows10_theme_path from bleachbit.Options import options from bleachbit.GuiPreferences import PreferencesDialog from bleachbit.Cleaner import backends, register_cleaners import bleachbit from gi.repository import Gtk, Gdk, GObject, GLib, Gio import glob import logging import os import sys import threading import time import gi gi.require_version('Gtk', '3.0') if os.name == 'nt': from bleachbit import Windows logger = logging.getLogger(__name__) def threaded(func): """Decoration to create a threaded function""" def wrapper(*args): thread = threading.Thread(target=func, args=args) thread.start() return wrapper def notify_gi(msg): """Show a pop-up notification. The Windows pygy-aio installer does not include notify, so this is just for Linux. """ gi.require_version('Notify', '0.7') from gi.repository import Notify if Notify.init(APP_NAME): notify = Notify.Notification.new('BleachBit', msg, 'bleachbit') notify.set_hint("desktop-entry", GLib.Variant('s', 'bleachbit')) notify.show() notify.set_timeout(10000) def notify_plyer(msg): """Show a pop-up notification. Linux distributions do not include plyer, so this is just for Windows. """ from bleachbit import bleachbit_exe_path # On Windows 10, PNG does not work. __icon_fns = ( os.path.normpath(os.path.join(bleachbit_exe_path, 'share\\bleachbit.ico')), os.path.normpath(os.path.join(bleachbit_exe_path, 'windows\\bleachbit.ico'))) icon_fn = None for __icon_fn in __icon_fns: if os.path.exists(__icon_fn): icon_fn = __icon_fn break from plyer import notification notification.notify( title=APP_NAME, message=msg, app_name=APP_NAME, # not shown on Windows 10 app_icon=icon_fn, ) def notify(msg): """Show a popup-notification""" import importlib if importlib.util.find_spec('plyer'): # On Windows, use Plyer. notify_plyer(msg) return # On Linux, use GTK Notify. notify_gi(msg) class Bleachbit(Gtk.Application): _window = None _shred_paths = None _auto_exit = False def __init__(self, uac=True, shred_paths=None, auto_exit=False): application_id_suffix = self._init_windows_misc(auto_exit, shred_paths, uac) application_id = '{}{}'.format('org.gnome.Bleachbit', application_id_suffix) Gtk.Application.__init__( self, application_id=application_id, flags=Gio.ApplicationFlags.FLAGS_NONE) GObject.threads_init() if auto_exit: # This is used for automated testing of whether the GUI can start. # It is called from assert_execute_console() in windows/setup_py2exe.py self._auto_exit = True if shred_paths: self._shred_paths = shred_paths if os.name == 'nt': # clean up nonce files https://github.com/bleachbit/bleachbit/issues/858 import atexit atexit.register(Windows.cleanup_nonce) # BitDefender false positive. BitDefender didn't mark BleachBit as infected or show # anything in its log, but sqlite would fail to import unless BitDefender was in "game mode." # https://www.bleachbit.org/forum/074-fails-errors try: import sqlite3 except ImportError: logger.exception( _("Error loading the SQLite module: the antivirus software may be blocking it.")) def _init_windows_misc(self, auto_exit, shred_paths, uac): application_id_suffix = '' is_context_menu_executed = auto_exit and shred_paths if os.name == 'nt': if Windows.elevate_privileges(uac): # privileges escalated in other process sys.exit(0) if is_context_menu_executed: # When we have a running application and executing the Windows # context menu command we start a new process with new application_id. # That is because the command line arguments of the context menu command # are not passed to the already running instance. application_id_suffix = 'ContextMenuShred' return application_id_suffix def build_app_menu(self): """Build the application menu On Linux with GTK 3.24, this code is necessary but not sufficient for the menu to work. The headerbar code is also needed. On Windows with GTK 3.18, this cde is sufficient for the menu to work. """ builder = Gtk.Builder() builder.add_from_file(bleachbit.app_menu_filename) menu = builder.get_object('app-menu') self.set_app_menu(menu) # set up mappings between in app-menu.ui and methods in this class actions = {'shredFiles': self.cb_shred_file, 'shredFolders': self.cb_shred_folder, 'shredClipboard': self.cb_shred_clipboard, 'wipeFreeSpace': self.cb_wipe_free_space, 'makeChaff': self.cb_make_chaff, 'shredQuit': self.cb_shred_quit, 'preferences': self.cb_preferences_dialog, 'systemInformation': self.system_information_dialog, 'help': self.cb_help, 'about': self.about} for action_name, callback in actions.items(): action = Gio.SimpleAction.new(action_name, None) action.connect('activate', callback) self.add_action(action) def cb_help(self, action, param): """Callback for help""" GuiBasic.open_url(bleachbit.help_contents_url, self._window) def cb_make_chaff(self, action, param): """Callback to make chaff""" from bleachbit.GuiChaff import ChaffDialog cd = ChaffDialog(self._window) cd.run() def cb_shred_file(self, action, param): """Callback for shredding a file""" # get list of files paths = GuiBasic.browse_files(self._window, _("Choose files to shred")) if not paths: return GUI.shred_paths(self._window, paths) def cb_shred_folder(self, action, param): """Callback for shredding a folder""" paths = GuiBasic.browse_folder(self._window, _("Choose folder to shred"), multiple=True, stock_button=_('_Delete')) if not paths: return GUI.shred_paths(self._window, paths) def cb_shred_clipboard(self, action, param): """Callback for menu option: shred paths from clipboard""" clipboard = Gtk.Clipboard.get(Gdk.SELECTION_CLIPBOARD) clipboard.request_targets(self.cb_clipboard_uri_received) def cb_clipboard_uri_received(self, clipboard, targets, data): """Callback for when URIs are received from clipboard""" shred_paths = None if Gdk.atom_intern_static_string('text/uri-list') in targets: # Linux shred_uris = clipboard.wait_for_contents( Gdk.atom_intern_static_string('text/uri-list')).get_uris() shred_paths = FileUtilities.uris_to_paths(shred_uris) elif Gdk.atom_intern_static_string('FileNameW') in targets: # Windows # Use non-GTK+ functions because because GTK+ 2 does not work. shred_paths = Windows.get_clipboard_paths() if shred_paths: GUI.shred_paths(self._window, shred_paths) else: logger.warning(_('No paths found in clipboard.')) def cb_shred_quit(self, action, param): """Shred settings (for privacy reasons) and quit""" # build a list of paths to delete paths = [] if os.name == 'nt' and portable_mode: # in portable mode on Windows, the options directory includes # executables paths.append(bleachbit.options_file) if os.path.isdir(bleachbit.personal_cleaners_dir): paths.append(bleachbit.personal_cleaners_dir) for f in glob.glob(os.path.join(bleachbit.options_dir, "*.bz2")): paths.append(f) else: paths.append(bleachbit.options_dir) # prompt the user to confirm if not GUI.shred_paths(self._window, paths, shred_settings=True): logger.debug('user aborted shred') # aborted return # Quit the application through the idle loop to allow the worker # to delete the files. Use the lowest priority because the worker # uses the standard priority. Otherwise, this will quit before # the files are deleted. # # Rebuild a minimal bleachbit.ini when quitting GLib.idle_add(self.quit, None, None, True, priority=GObject.PRIORITY_LOW) def cb_wipe_free_space(self, action, param): """callback to wipe free space in arbitrary folder""" path = GuiBasic.browse_folder(self._window, _("Choose a folder"), multiple=False, stock_button=_('_OK')) if not path: # user cancelled return backends['_gui'] = Cleaner.create_wipe_cleaner(path) # execute operations = {'_gui': ['free_disk_space']} self._window.preview_or_run_operations(True, operations) def get_preferences_dialog(self): return self._window.get_preferences_dialog() def cb_preferences_dialog(self, action, param): """Callback for preferences dialog""" pref = self.get_preferences_dialog() pref.run() # In case the user changed the log level... GUI.update_log_level(self._window) def get_about_dialog(self): dialog = Gtk.AboutDialog(comments='Program to clean unnecessary files', copyright='Copyright (C) 2008-2021 Andrew Ziem', program_name=APP_NAME, version=bleachbit.APP_VERSION, website=bleachbit.APP_URL, transient_for=self._window) try: with open(bleachbit.license_filename) as f_license: dialog.set_license(f_license.read()) except (IOError, TypeError): dialog.set_license( _("GNU General Public License version 3 or later.\nSee https://www.gnu.org/licenses/gpl-3.0.txt")) # dialog.set_name(APP_NAME) # TRANSLATORS: Maintain the names of translators here. # Launchpad does this automatically for translations # typed in Launchpad. This is a special string shown # in the 'About' box. dialog.set_translator_credits(_("translator-credits")) if appicon_path and os.path.exists(appicon_path): icon = Gtk.Image.new_from_file(appicon_path) dialog.set_logo(icon.get_pixbuf()) return dialog def about(self, _action, _param): """Create and show the about dialog""" dialog = self.get_about_dialog() dialog.run() dialog.destroy() def do_startup(self): Gtk.Application.do_startup(self) self.build_app_menu() def quit(self, _action=None, _param=None, init_configuration=False): if init_configuration: bleachbit.Options.init_configuration() self._window.destroy() def get_system_information_dialog(self): """Show system information dialog""" dialog = Gtk.Dialog(_("System information"), self._window) dialog.set_default_size(600, 400) txtbuffer = Gtk.TextBuffer() from bleachbit import SystemInformation txt = SystemInformation.get_system_information() txtbuffer.set_text(txt) textview = Gtk.TextView.new_with_buffer(txtbuffer) textview.set_editable(False) swindow = Gtk.ScrolledWindow() swindow.set_policy(Gtk.PolicyType.AUTOMATIC, Gtk.PolicyType.AUTOMATIC) swindow.add(textview) dialog.vbox.pack_start(swindow, True, True, 0) dialog.add_buttons(Gtk.STOCK_COPY, 100, Gtk.STOCK_CLOSE, Gtk.ResponseType.CLOSE) return (dialog, txt) def system_information_dialog(self, _action, _param): dialog, txt = self.get_system_information_dialog() dialog.show_all() while True: rc = dialog.run() if rc != 100: break clipboard = Gtk.Clipboard.get(Gdk.SELECTION_CLIPBOARD) clipboard.set_text(txt, -1) dialog.destroy() def do_activate(self): if not self._window: self._window = GUI( application=self, title=APP_NAME, auto_exit=self._auto_exit) self._window.present() if self._shred_paths: GLib.idle_add(GUI.shred_paths, self._window, self._shred_paths, priority=GObject.PRIORITY_LOW) # When we shred paths and auto exit with the Windows Explorer context menu command we close the # application in GUI.shred_paths, because if it is closed from here there are problems. # Most probably this is something related with how GTK handles idle quit calls. elif self._auto_exit: GLib.idle_add(self.quit, priority=GObject.PRIORITY_LOW) print('Success') class TreeInfoModel: """Model holds information to be displayed in the tree view""" def __init__(self): self.tree_store = Gtk.TreeStore( GObject.TYPE_STRING, GObject.TYPE_BOOLEAN, GObject.TYPE_PYOBJECT, GObject.TYPE_STRING) if not self.tree_store: raise Exception("cannot create tree store") self.row_changed_handler_id = None self.refresh_rows() self.tree_store.set_sort_func(3, self.sort_func) self.tree_store.set_sort_column_id(3, Gtk.SortType.ASCENDING) def get_model(self): """Return the tree store""" return self.tree_store def on_row_changed(self, __treemodel, path, __iter): """Event handler for when a row changes""" parent = self.tree_store[path[0]][2] child = None if len(path) == 2: child = self.tree_store[path][2] value = self.tree_store[path][1] options.set_tree(parent, child, value) def refresh_rows(self): """Clear rows (cleaners) and add them fresh""" if self.row_changed_handler_id: self.tree_store.disconnect(self.row_changed_handler_id) self.tree_store.clear() for key in sorted(backends): if not any(backends[key].get_options()): # localizations has no options, so it should be hidden # https://github.com/az0/bleachbit/issues/110 continue c_name = backends[key].get_name() c_id = backends[key].get_id() c_value = options.get_tree(c_id, None) if not c_value and options.get('auto_hide') and backends[key].auto_hide(): logger.debug("automatically hiding cleaner '%s'", c_id) continue parent = self.tree_store.append(None, (c_name, c_value, c_id, "")) for (o_id, o_name) in backends[key].get_options(): o_value = options.get_tree(c_id, o_id) self.tree_store.append(parent, (o_name, o_value, o_id, "")) self.row_changed_handler_id = self.tree_store.connect("row-changed", self.on_row_changed) def sort_func(self, model, iter1, iter2, _user_data): """Sort the tree by the display name""" value1 = model[iter1][0].lower() value2 = model[iter2][0].lower() if value1 == value2: return 0 if value1 > value2: return 1 return -1 class TreeDisplayModel: """Displays the info model in a view""" def make_view(self, model, parent, context_menu_event): """Create and return a TreeView object""" self.view = Gtk.TreeView.new_with_model(model) # hide headers self.view.set_headers_visible(False) # listen for right click (context menu) self.view.connect("button_press_event", context_menu_event) # first column self.renderer0 = Gtk.CellRendererText() self.column0 = Gtk.TreeViewColumn(_("Name"), self.renderer0, text=0) self.view.append_column(self.column0) self.view.set_search_column(0) # second column self.renderer1 = Gtk.CellRendererToggle() self.renderer1.set_property('activatable', True) self.renderer1.connect('toggled', self.col1_toggled_cb, model, parent) self.column1 = Gtk.TreeViewColumn(_("Active"), self.renderer1) self.column1.add_attribute(self.renderer1, "active", 1) self.view.append_column(self.column1) # third column self.renderer2 = Gtk.CellRendererText() self.renderer2.set_alignment(1.0, 0.0) # TRANSLATORS: Size is the label for the column that shows how # much space an option would clean or did clean self.column2 = Gtk.TreeViewColumn(_("Size"), self.renderer2, text=3) self.column2.set_alignment(1.0) self.view.append_column(self.column2) # finish self.view.expand_all() return self.view def set_cleaner(self, path, model, parent_window, value): """Activate or deactivate option of cleaner.""" assert isinstance(value, bool) assert isinstance(model, Gtk.TreeStore) cleaner_id = None i = path if isinstance(i, str): # type is either str or gtk.TreeIter i = model.get_iter(path) parent = model.iter_parent(i) if parent: # this is an option (child), not a cleaner (parent) cleaner_id = model[parent][2] option_id = model[path][2] if cleaner_id and value: # When enabling an option, present any warnings. # (When disabling an option, there is no need to present warnings.) warning = backends[cleaner_id].get_warning(option_id) # TRANSLATORS: %(cleaner) may be Firefox, System, etc. # %(option) may be cache, logs, cookies, etc. # %(warning) may be 'This option is really slow' msg = _("Warning regarding %(cleaner)s - %(option)s:\n\n%(warning)s") % \ {'cleaner': model[parent][0], 'option': model[path][0], 'warning': warning} if warning: resp = GuiBasic.message_dialog(parent_window, msg, Gtk.MessageType.WARNING, Gtk.ButtonsType.OK_CANCEL) if Gtk.ResponseType.OK != resp: # user cancelled, so don't toggle option return model[path][1] = value def col1_toggled_cb(self, cell, path, model, parent_window): """Callback for toggling cleaners""" is_toggled_on = not model[path][1] # Is the new state enabled? self.set_cleaner(path, model, parent_window, is_toggled_on) i = model.get_iter(path) parent = model.iter_parent(i) if parent and is_toggled_on: # If child is enabled, then also enable the parent. model[parent][1] = True # If all siblings were toggled off, then also disable the parent. if parent and not is_toggled_on: sibling = model.iter_nth_child(parent, 0) any_sibling_enabled = False while sibling: if model[sibling][1]: any_sibling_enabled = True sibling = model.iter_next(sibling) if not any_sibling_enabled: model[parent][1] = False # If toggled and has children, then do the same for each child. child = model.iter_children(i) while child: self.set_cleaner(child, model, parent_window, is_toggled_on) child = model.iter_next(child) return class GUI(Gtk.ApplicationWindow): """The main application GUI""" _style_provider = None _style_provider_regular = None _style_provider_dark = None def __init__(self, auto_exit, *args, **kwargs): super(GUI, self).__init__(*args, **kwargs) self._show_splash_screen() self._auto_exit = auto_exit self.set_wmclass(APP_NAME, APP_NAME) self.populate_window() # Redirect logging to the GUI. bb_logger = logging.getLogger('bleachbit') from bleachbit.Log import GtkLoggerHandler self.gtklog = GtkLoggerHandler(self.append_text) bb_logger.addHandler(self.gtklog) # process any delayed logs from bleachbit.Log import DelayLog if isinstance(sys.stderr, DelayLog): for msg in sys.stderr.read(): self.append_text(msg) # if stderr was redirected - keep redirecting it sys.stderr = self.gtklog self.set_windows10_theme() Gtk.Settings.get_default().set_property( 'gtk-application-prefer-dark-theme', options.get('dark_mode')) if options.is_corrupt(): logger.error( _('Resetting the configuration file because it is corrupt: %s') % bleachbit.options_file) bleachbit.Options.init_configuration() GLib.idle_add(self.cb_refresh_operations) def _show_splash_screen(self): if os.name != 'nt': return font_conf_file = Windows.get_font_conf_file() if not os.path.exists(font_conf_file): logger.error('No fonts.conf file') return has_cache = Windows.has_fontconfig_cache(font_conf_file) if not has_cache: Windows.splash_thread.start() def _confirm_delete(self, mention_preview, shred_settings=False): if options.get("delete_confirmation"): return GuiBasic.delete_confirmation_dialog(self, mention_preview, shred_settings=shred_settings) return True def get_preferences_dialog(self): return PreferencesDialog( self, self.cb_refresh_operations, self.set_windows10_theme) def shred_paths(self, paths, shred_settings=False): """Shred file or folders When shredding_settings=True: If user confirms to delete, then returns True. If user aborts, returns False. """ # create a temporary cleaner object backends['_gui'] = Cleaner.create_simple_cleaner(paths) # preview and confirm operations = {'_gui': ['files']} self.preview_or_run_operations(False, operations) if self._confirm_delete(False, shred_settings): # delete self.preview_or_run_operations(True, operations) if shred_settings: return True if self._auto_exit: GLib.idle_add(self.close, priority=GObject.PRIORITY_LOW) # user aborted return False def append_text(self, text, tag=None, __iter=None, scroll=True): """Add some text to the main log""" if not __iter: __iter = self.textbuffer.get_end_iter() if tag: self.textbuffer.insert_with_tags_by_name(__iter, text, tag) else: self.textbuffer.insert(__iter, text) # Scroll to end. If the command is run directly instead of # through the idle loop, it may only scroll most of the way # as seen on Ubuntu 9.04 with Italian and Spanish. if scroll: GLib.idle_add(lambda: self.textview.scroll_mark_onscreen( self.textbuffer.get_insert())) def update_log_level(self): """This gets called when the log level might have changed via the preferences.""" self.gtklog.update_log_level() def on_selection_changed(self, selection): """When the tree view selection changed""" model = self.view.get_model() selected_rows = selection.get_selected_rows() if not selected_rows[1]: # empty # happens when searching in the tree view return paths = selected_rows[1][0] row = paths[0] name = model[row][0] cleaner_id = model[row][2] self.progressbar.hide() description = backends[cleaner_id].get_description() self.textbuffer.set_text("") self.append_text(name + "\n", 'operation', scroll=False) if not description: description = "" self.append_text(description + "\n\n\n", 'description', scroll=False) for (label, description) in backends[cleaner_id].get_option_descriptions(): self.append_text(label, 'option_label', scroll=False) if description: self.append_text(': ', 'option_label', scroll=False) self.append_text(description, scroll=False) self.append_text("\n\n", scroll=False) def get_selected_operations(self): """Return a list of the IDs of the selected operations in the tree view""" ret = [] model = self.tree_store.get_model() path = Gtk.TreePath(0) __iter = model.get_iter(path) while __iter: if model[__iter][1]: ret.append(model[__iter][2]) __iter = model.iter_next(__iter) return ret def get_operation_options(self, operation): """For the given operation ID, return a list of the selected option IDs.""" ret = [] model = self.tree_store.get_model() path = Gtk.TreePath(0) __iter = model.get_iter(path) while __iter: if operation == model[__iter][2]: iterc = model.iter_children(__iter) if not iterc: return None while iterc: if model[iterc][1]: # option is enabled ret.append(model[iterc][2]) iterc = model.iter_next(iterc) return ret __iter = model.iter_next(__iter) return None def set_sensitive(self, is_sensitive): """Disable commands while an operation is running""" self.view.set_sensitive(is_sensitive) self.preview_button.set_sensitive(is_sensitive) self.run_button.set_sensitive(is_sensitive) self.stop_button.set_sensitive(not is_sensitive) def run_operations(self, __widget): """Event when the 'delete' toolbar button is clicked.""" # fixme: should present this dialog after finding operations # Disable delete confirmation message. # if the option is selected under preference. if self._confirm_delete(True): self.preview_or_run_operations(True) def preview_or_run_operations(self, really_delete, operations=None): """Preview operations or run operations (delete files)""" assert isinstance(really_delete, bool) from bleachbit import Worker self.start_time = None if not operations: operations = { operation: self.get_operation_options(operation) for operation in self.get_selected_operations() } assert isinstance(operations, dict) if not operations: # empty GuiBasic.message_dialog(self, _("You must select an operation"), Gtk.MessageType.WARNING, Gtk.ButtonsType.OK) return try: self.set_sensitive(False) self.textbuffer.set_text("") self.progressbar.show() self.worker = Worker.Worker(self, really_delete, operations) except Exception: logger.exception('Error in Worker()') else: self.start_time = time.time() worker = self.worker.run() GLib.idle_add(worker.__next__) def worker_done(self, worker, really_delete): """Callback for when Worker is done""" self.progressbar.set_text("") self.progressbar.set_fraction(1) self.progressbar.set_text(_("Done.")) self.textview.scroll_mark_onscreen(self.textbuffer.get_insert()) self.set_sensitive(True) # Close the program after cleaning is completed. # if the option is selected under preference. if really_delete: if options.get("exit_done"): sys.exit() # notification for long-running process elapsed = (time.time() - self.start_time) logger.debug('elapsed time: %d seconds', elapsed) if elapsed < 10 or self.is_active(): return notify(_("Done.")) def create_operations_box(self): """Create and return the operations box (which holds a tree view)""" scrolled_window = Gtk.ScrolledWindow() scrolled_window.set_policy( Gtk.PolicyType.NEVER, Gtk.PolicyType.AUTOMATIC) scrolled_window.set_overlay_scrolling(False) self.tree_store = TreeInfoModel() display = TreeDisplayModel() mdl = self.tree_store.get_model() self.view = display.make_view( mdl, self, self.context_menu_event) self.view.get_selection().connect("changed", self.on_selection_changed) scrollbar_width = scrolled_window.get_vscrollbar().get_preferred_width()[1] self.view.set_margin_right(scrollbar_width) # avoid conflict with scrollbar scrolled_window.add(self.view) return scrolled_window def cb_refresh_operations(self): """Callback to refresh the list of cleaners""" # Is this the first time in this session? if not hasattr(self, 'recognized_cleanerml') and not self._auto_exit: from bleachbit import RecognizeCleanerML RecognizeCleanerML.RecognizeCleanerML() self.recognized_cleanerml = True # reload cleaners from disk self.view.expand_all() self.progressbar.show() rc = register_cleaners(self.update_progress_bar, self.cb_register_cleaners_done) GLib.idle_add(rc.__next__) return False def cb_register_cleaners_done(self): """Called from register_cleaners()""" self.progressbar.hide() # update tree view self.tree_store.refresh_rows() # expand tree view self.view.expand_all() # Check for online updates. if not self._auto_exit and \ bleachbit.online_update_notification_enabled and \ options.get("check_online_updates") and \ not hasattr(self, 'checked_for_updates'): self.checked_for_updates = True self.check_online_updates() # Show information for first start. # (The first start flag is set also for each new version.) if options.get("first_start") and not self._auto_exit: if os.name == 'posix': self.append_text( _('Access the application menu by clicking the hamburger icon on the title bar.')) pref = self.get_preferences_dialog() pref.run() elif os.name == 'nt': self.append_text( _('Access the application menu by clicking the logo on the title bar.')) options.set('first_start', False) if os.name == 'nt': # BitDefender false positive. BitDefender didn't mark BleachBit as infected or show # anything in its log, but sqlite would fail to import unless BitDefender was in "game mode." # http://bleachbit.sourceforge.net/forum/074-fails-errors try: import sqlite3 except ImportError as e: self.append_text( _("Error loading the SQLite module: the antivirus software may be blocking it."), 'error') # Show notice about admin privileges. if os.name == 'posix' and os.path.expanduser('~') == '/root': self.append_text( _('You are running BleachBit with administrative privileges for cleaning shared parts of the system, and references to the user profile folder will clean only the root account.')+'\n') if os.name == 'nt' and options.get('shred'): from win32com.shell.shell import IsUserAnAdmin if not IsUserAnAdmin(): self.append_text( _('Run BleachBit with administrator privileges to improve the accuracy of overwriting the contents of files.')) self.append_text('\n') # remove from idle loop (see GObject.idle_add) return False def cb_run_option(self, widget, really_delete, cleaner_id, option_id): """Callback from context menu to delete/preview a single option""" operations = {cleaner_id: [option_id]} # preview if not really_delete: self.preview_or_run_operations(False, operations) return # delete if self._confirm_delete(False): self.preview_or_run_operations(True, operations) return def cb_stop_operations(self, __widget): """Callback to stop the preview/cleaning process""" self.worker.abort() def context_menu_event(self, treeview, event): """When user right clicks on the tree view""" if event.button != 3: return False pathinfo = treeview.get_path_at_pos(int(event.x), int(event.y)) if not pathinfo: return False path, col, _cellx, _celly = pathinfo treeview.grab_focus() treeview.set_cursor(path, col, 0) # context menu applies only to children, not parents if len(path) != 2: return False # find the selected option model = treeview.get_model() option_id = model[path][2] cleaner_id = model[path[0]][2] # make a menu menu = Gtk.Menu() menu.connect('hide', lambda widget: widget.detach()) # TRANSLATORS: this is the context menu preview_item = Gtk.MenuItem(label=_("Preview")) preview_item.connect('activate', self.cb_run_option, False, cleaner_id, option_id) menu.append(preview_item) # TRANSLATORS: this is the context menu clean_item = Gtk.MenuItem(label=_("Clean")) clean_item.connect('activate', self.cb_run_option, True, cleaner_id, option_id) menu.append(clean_item) # show the context menu menu.attach_to_widget(treeview) menu.show_all() menu.popup(None, None, None, None, event.button, event.time) return True def setup_drag_n_drop(self): def cb_drag_data_received(widget, _context, _x, _y, data, info, _time): if info == 80: uris = data.get_uris() paths = FileUtilities.uris_to_paths(uris) self.shred_paths(paths) def setup_widget(widget): widget.drag_dest_set(Gtk.DestDefaults.MOTION | Gtk.DestDefaults.HIGHLIGHT | Gtk.DestDefaults.DROP, [Gtk.TargetEntry.new("text/uri-list", 0, 80)], Gdk.DragAction.COPY) widget.connect('drag_data_received', cb_drag_data_received) setup_widget(self) setup_widget(self.textview) self.textview.connect('drag_motion', lambda widget, context, x, y, time: True) def update_progress_bar(self, status): """Callback to update the progress bar with number or text""" if isinstance(status, float): self.progressbar.set_fraction(status) elif isinstance(status, str): self.progressbar.set_show_text(True) self.progressbar.set_text(status) else: raise RuntimeError('unexpected type: ' + str(type(status))) def update_item_size(self, option, option_id, bytes_removed): """Update size in tree control""" model = self.view.get_model() text = FileUtilities.bytes_to_human(bytes_removed) if bytes_removed == 0: text = "" treepath = Gtk.TreePath(0) try: __iter = model.get_iter(treepath) except ValueError as e: logger.warning( 'ValueError in get_iter() when updating file size for tree path=%s' % treepath) return while __iter: if model[__iter][2] == option: if option_id == -1: model[__iter][3] = text else: child = model.iter_children(__iter) while child: if model[child][2] == option_id: model[child][3] = text child = model.iter_next(child) __iter = model.iter_next(__iter) def update_total_size(self, bytes_removed): """Callback to update the total size cleaned""" context_id = self.status_bar.get_context_id('size') text = FileUtilities.bytes_to_human(bytes_removed) if bytes_removed == 0: text = "" self.status_bar.push(context_id, text) def create_headerbar(self): """Create the headerbar""" hbar = Gtk.HeaderBar() hbar.props.show_close_button = True hbar.props.title = APP_NAME box = Gtk.Box() Gtk.StyleContext.add_class(box.get_style_context(), "linked") if os.name == 'nt': icon_size = Gtk.IconSize.BUTTON else: icon_size = Gtk.IconSize.LARGE_TOOLBAR # create the preview button self.preview_button = Gtk.Button.new_from_icon_name( 'edit-find', icon_size) self.preview_button.set_always_show_image(True) self.preview_button.connect( 'clicked', lambda *dummy: self.preview_or_run_operations(False)) self.preview_button.set_tooltip_text( _("Preview files in the selected operations (without deleting any files)")) # TRANSLATORS: This is the preview button on the main window. It # previews changes. self.preview_button.set_label(_('Preview')) box.add(self.preview_button) # create the delete button self.run_button = Gtk.Button.new_from_icon_name( 'edit-clear-all', icon_size) self.run_button.set_always_show_image(True) # TRANSLATORS: This is the clean button on the main window. # It makes permanent changes: usually deleting files, sometimes # altering them. self.run_button.set_label(_('Clean')) self.run_button.set_tooltip_text( _("Clean files in the selected operations")) self.run_button.connect("clicked", self.run_operations) box.add(self.run_button) # stop cleaning self.stop_button = Gtk.Button.new_from_icon_name( 'process-stop', icon_size) self.stop_button.set_always_show_image(True) self.stop_button.set_label(_('Abort')) self.stop_button.set_tooltip_text( _('Abort the preview or cleaning process')) self.stop_button.set_sensitive(False) self.stop_button.connect('clicked', self.cb_stop_operations) box.add(self.stop_button) hbar.pack_start(box) # Add hamburger menu on the right. # This is not needed for Microsoft Windows because other code places its # menu on the left side. if os.name == 'nt': return hbar menu_button = Gtk.MenuButton() icon = Gio.ThemedIcon(name="open-menu-symbolic") image = Gtk.Image.new_from_gicon(icon, Gtk.IconSize.BUTTON) builder = Gtk.Builder() builder.add_from_file(bleachbit.app_menu_filename) menu_button.set_menu_model(builder.get_object('app-menu')) menu_button.add(image) hbar.pack_end(menu_button) return hbar def on_configure_event(self, widget, event): (x, y) = self.get_position() (width, height) = self.get_size() # fixup maximized window position: # on Windows if a window is maximized on a secondary monitor it is moved off the screen if 'nt' == os.name: window = self.get_window() if window.get_state() & Gdk.WindowState.MAXIMIZED != 0: screen = self.get_screen() monitor_num = screen.get_monitor_at_window(window) g = screen.get_monitor_geometry(monitor_num) if x < g.x or x >= g.x + g.width or y < g.y or y >= g.y + g.height: logger.debug("Maximized window {}+{}: monitor ({}) geometry = {}+{}".format( (x, y), (width, height), monitor_num, (g.x, g.y), (g.width, g.height))) self.move(g.x, g.y) return True # save window position and size options.set("window_x", x, commit=False) options.set("window_y", y, commit=False) options.set("window_width", width, commit=False) options.set("window_height", height, commit=False) return False def on_window_state_event(self, widget, event): # save window state fullscreen = event.new_window_state & Gdk.WindowState.FULLSCREEN != 0 options.set("window_fullscreen", fullscreen, commit=False) maximized = event.new_window_state & Gdk.WindowState.MAXIMIZED != 0 options.set("window_maximized", maximized, commit=False) return False def on_delete_event(self, widget, event): # commit options to disk options.commit() return False def on_show(self, widget): if 'nt' == os.name and Windows.splash_thread.is_alive(): Windows.splash_thread.join(0) # restore window position, size and state if not options.get('remember_geometry'): return if options.has_option("window_x") and options.has_option("window_y") and \ options.has_option("window_width") and options.has_option("window_height"): r = Gdk.Rectangle() (r.x, r.y) = (options.get("window_x"), options.get("window_y")) (r.width, r.height) = (options.get( "window_width"), options.get("window_height")) screen = self.get_screen() monitor_num = screen.get_monitor_at_point(r.x, r.y) g = screen.get_monitor_geometry(monitor_num) # only restore position and size if window left corner # is within the closest monitor if r.x >= g.x and r.x < g.x + g.width and \ r.y >= g.y and r.y < g.y + g.height: logger.debug("closest monitor ({}) geometry = {}+{}, window geometry = {}+{}".format( monitor_num, (g.x, g.y), (g.width, g.height), (r.x, r.y), (r.width, r.height))) self.move(r.x, r.y) self.resize(r.width, r.height) if options.get("window_fullscreen"): self.fullscreen() elif options.get("window_maximized"): self.maximize() def set_windows10_theme(self): """Toggle the Windows 10 theme""" if not 'nt' == os.name: return if not self._style_provider_regular: self._style_provider_regular = Gtk.CssProvider() self._style_provider_regular.load_from_path( os.path.join(windows10_theme_path, 'gtk.css')) if not self._style_provider_dark: self._style_provider_dark = Gtk.CssProvider() self._style_provider_dark.load_from_path( os.path.join(windows10_theme_path, 'gtk-dark.css')) screen = Gdk.Display.get_default_screen(Gdk.Display.get_default()) if self._style_provider is not None: Gtk.StyleContext.remove_provider_for_screen( screen, self._style_provider) if options.get("win10_theme"): if options.get("dark_mode"): self._style_provider = self._style_provider_dark else: self._style_provider = self._style_provider_regular Gtk.StyleContext.add_provider_for_screen( screen, self._style_provider, Gtk.STYLE_PROVIDER_PRIORITY_APPLICATION) else: self._style_provider = None def populate_window(self): """Create the main application window""" screen = self.get_screen() self.set_default_size(min(screen.width(), 800), min(screen.height(), 600)) self.set_position(Gtk.WindowPosition.CENTER) self.connect("configure-event", self.on_configure_event) self.connect("window-state-event", self.on_window_state_event) self.connect("delete-event", self.on_delete_event) self.connect("show", self.on_show) if appicon_path and os.path.exists(appicon_path): self.set_icon_from_file(appicon_path) # add headerbar self.headerbar = self.create_headerbar() self.set_titlebar(self.headerbar) # split main window twice hbox = Gtk.Box(homogeneous=False) vbox = Gtk.Box(orientation=Gtk.Orientation.VERTICAL, homogeneous=False) self.add(vbox) vbox.add(hbox) # add operations to left operations = self.create_operations_box() hbox.pack_start(operations, False, True, 0) # create the right side of the window right_box = Gtk.Box(orientation=Gtk.Orientation.VERTICAL) self.progressbar = Gtk.ProgressBar() right_box.pack_start(self.progressbar, False, True, 0) # add output display on right self.textbuffer = Gtk.TextBuffer() swindow = Gtk.ScrolledWindow() swindow.set_policy(Gtk.PolicyType.AUTOMATIC, Gtk.PolicyType.AUTOMATIC) swindow.set_property('expand', True) self.textview = Gtk.TextView.new_with_buffer(self.textbuffer) self.textview.set_editable(False) self.textview.set_wrap_mode(Gtk.WrapMode.WORD) swindow.add(self.textview) right_box.add(swindow) hbox.add(right_box) # add markup tags tt = self.textbuffer.get_tag_table() style_operation = Gtk.TextTag.new('operation') style_operation.set_property('size-points', 14) style_operation.set_property('weight', 700) style_operation.set_property('pixels-above-lines', 10) style_operation.set_property('justification', Gtk.Justification.CENTER) tt.add(style_operation) style_description = Gtk.TextTag.new('description') style_description.set_property( 'justification', Gtk.Justification.CENTER) tt.add(style_description) style_option_label = Gtk.TextTag.new('option_label') style_option_label.set_property('weight', 700) style_option_label.set_property('left-margin', 20) tt.add(style_option_label) style_operation = Gtk.TextTag.new('error') style_operation.set_property('foreground', '#b00000') tt.add(style_operation) self.status_bar = Gtk.Statusbar() vbox.add(self.status_bar) # setup drag&drop self.setup_drag_n_drop() # done self.show_all() self.progressbar.hide() @threaded def check_online_updates(self): """Check for software updates in background""" from bleachbit import Update try: updates = Update.check_updates(options.get('check_beta'), options.get('update_winapp2'), self.append_text, lambda: GLib.idle_add(self.cb_refresh_operations)) if updates: GLib.idle_add( lambda: Update.update_dialog(self, updates)) except Exception: logger.exception(_("Error when checking for updates: ")) bleachbit-4.4.2/bleachbit/Memory.py0000775000175000017500000002546214144024253016134 0ustar fabiofabio# vim: ts=4:sw=4:expandtab # -*- coding: UTF-8 -*- # BleachBit # Copyright (C) 2008-2021 Andrew Ziem # https://www.bleachbit.org # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . """ Wipe memory """ from bleachbit import FileUtilities from bleachbit import General from bleachbit import _ import logging import os import re import subprocess import sys logger = logging.getLogger(__name__) def count_swap_linux(): """Count the number of swap devices in use""" count = 0 with open("/proc/swaps") as f: for line in f: if line[0] == '/': count += 1 return count def get_proc_swaps(): """Return the output of 'swapon -s'""" # Usually 'swapon -s' is identical to '/proc/swaps' # Here is one exception: # https://bugs.launchpad.net/ubuntu/+source/bleachbit/+bug/1092792 (rc, stdout, _stderr) = General.run_external(['swapon', '-s']) if 0 == rc: return stdout logger.debug( _("The command 'swapoff -s' failed, so falling back to /proc/swaps for swap information.")) return open("/proc/swaps").read() def parse_swapoff(swapoff): """Parse the output of swapoff and return the device name""" # English is 'swapoff on /dev/sda5' but German is 'swapoff für ...' # Example output in English with LVM and hyphen: 'swapoff on /dev/mapper/lubuntu-swap_1' # This matches swap devices and swap files ret = re.search('^swapoff (\w* )?(/[\w/.-]+)$', swapoff) if not ret: # no matches return None return ret.group(2) def disable_swap_linux(): """Disable Linux swap and return list of devices""" if 0 == count_swap_linux(): return logger.debug(_("Disabling swap.")) args = ["swapoff", "-a", "-v"] (rc, stdout, stderr) = General.run_external(args) if 0 != rc: raise RuntimeError(stderr.replace("\n", "")) devices = [] for line in stdout.split('\n'): line = line.replace('\n', '') if '' == line: continue ret = parse_swapoff(line) if ret is None: raise RuntimeError("Unexpected output:\nargs='%(args)s'\nstdout='%(stdout)s'\nstderr='%(stderr)s'" % {'args': str(args), 'stdout': stdout, 'stderr': stderr}) devices.append(ret) return devices def enable_swap_linux(): """Enable Linux swap""" logger.debug(_("Re-enabling swap.")) args = ["swapon", "-a"] p = subprocess.Popen(args, stderr=subprocess.PIPE) p.wait() outputs = p.communicate() if 0 != p.returncode: raise RuntimeError(outputs[1].replace("\n", "")) def make_self_oom_target_linux(): """Make the current process the primary target for Linux out-of-memory killer""" # In Linux 2.6.36 the system changed from oom_adj to oom_score_adj path = '/proc/%d/oom_score_adj' % os.getpid() if os.path.exists(path): with open(path, 'w') as f: f.write('1000') else: path = '/proc/%d/oomadj' % os.getpid() if os.path.exists(path): with open(path, 'w') as f: f.write('15') # OOM likes nice processes logger.debug(_("Setting nice value %d for this process."), os.nice(19)) # OOM prefers non-privileged processes try: uid = General.getrealuid() if uid > 0: logger.debug( _("Dropping privileges of process ID {pid} to user ID {uid}.").format(pid=os.getpid(), uid=uid)) os.seteuid(uid) except: logger.exception('Error when dropping privileges') def fill_memory_linux(): """Fill unallocated memory""" report_free() allocbytes = int(physical_free() * 0.4) if allocbytes < 1024: return bytes_str = FileUtilities.bytes_to_human(allocbytes) # TRANSLATORS: The variable is a quantity like 5kB logger.info(_("Allocating and wiping %s of memory."), bytes_str) try: buf = '\x00' * allocbytes except MemoryError: pass else: fill_memory_linux() # TRANSLATORS: The variable is a quantity like 5kB logger.debug(_("Freeing %s of memory."), bytes_str) del buf report_free() def get_swap_size_linux(device, proc_swaps=None): """Return the size of the partition in bytes""" if proc_swaps is None: proc_swaps = get_proc_swaps() line = proc_swaps.split('\n')[0] if not re.search('Filename\s+Type\s+Size', line): raise RuntimeError("Unexpected first line in swap summary '%s'" % line) for line in proc_swaps.split('\n')[1:]: ret = re.search("%s\s+\w+\s+([0-9]+)\s" % device, line) if ret: return int(ret.group(1)) * 1024 raise RuntimeError("error: cannot find size of swap device '%s'\n%s" % (device, proc_swaps)) def get_swap_uuid(device): """Find the UUID for the swap device""" uuid = None args = ['blkid', device, '-s', 'UUID'] (_rc, stdout, _stderr) = General.run_external(args) for line in stdout.split('\n'): # example: /dev/sda5: UUID="ee0e85f6-6e5c-42b9-902f-776531938bbf" ret = re.search("^%s: UUID=\"([a-z0-9-]+)\"" % device, line) if ret is not None: uuid = ret.group(1) logger.debug(_("Found UUID for swap file {device} is {uuid}.").format( device=device, uuid=uuid)) return uuid def physical_free_darwin(run_vmstat=None): def parse_line(k, v): return k, int(v.strip(" .")) def get_page_size(line): m = re.match( r"Mach Virtual Memory Statistics: \(page size of (\d+) bytes\)", line) if m is None: raise RuntimeError("Can't parse vm_stat output") return int(m.groups()[0]) if run_vmstat is None: def run_vmstat(): return subprocess.check_output(["vm_stat"]) output = iter(run_vmstat().split("\n")) page_size = get_page_size(next(output)) vm_stat = dict(parse_line(*l.split(":")) for l in output if l != "") return vm_stat["Pages free"] * page_size def physical_free_linux(): """Return the physical free memory on Linux""" free_bytes = 0 with open("/proc/meminfo") as f: for line in f: line = line.replace("\n", "") ret = re.search('(MemFree|Cached):[ ]*([0-9]*) kB', line) if ret is not None: kb = int(ret.group(2)) free_bytes += kb * 1024 if free_bytes > 0: return free_bytes else: raise Exception("unknown") def physical_free_windows(): """Return physical free memory on Windows""" from ctypes import c_long, c_ulonglong from ctypes.wintypes import Structure, sizeof, windll, byref class MEMORYSTATUSEX(Structure): _fields_ = [ ('dwLength', c_long), ('dwMemoryLoad', c_long), ('ullTotalPhys', c_ulonglong), ('ullAvailPhys', c_ulonglong), ('ullTotalPageFile', c_ulonglong), ('ullAvailPageFile', c_ulonglong), ('ullTotalVirtual', c_ulonglong), ('ullAvailVirtual', c_ulonglong), ('ullExtendedVirtual', c_ulonglong), ] def GlobalMemoryStatusEx(): x = MEMORYSTATUSEX() x.dwLength = sizeof(x) windll.kernel32.GlobalMemoryStatusEx(byref(x)) return x z = GlobalMemoryStatusEx() return z.ullAvailPhys def physical_free(): if sys.platform.startswith('linux'): return physical_free_linux() elif 'win32' == sys.platform: return physical_free_windows() elif 'darwin' == sys.platform: return physical_free_darwin() else: raise RuntimeError('unsupported platform for physical_free()') def report_free(): """Report free memory""" bytes_free = physical_free() bytes_str = FileUtilities.bytes_to_human(bytes_free) # TRANSLATORS: The variable is a quantity like 5kB logger.debug(_("Physical free memory is %s."), bytes_str) def wipe_swap_linux(devices, proc_swaps): """Shred the Linux swap file and then reinitialize it""" if devices is None: return if 0 < count_swap_linux(): raise RuntimeError('Cannot wipe swap while it is in use') for device in devices: # if '/cryptswap' in device: # logger.info('Skipping encrypted swap device %s.', device) # continue # TRANSLATORS: The variable is a device like /dev/sda2 logger.info(_("Wiping the swap device %s."), device) safety_limit_bytes = 29 * 1024 ** 3 # 29 gibibytes actual_size_bytes = get_swap_size_linux(device, proc_swaps) if actual_size_bytes > safety_limit_bytes: raise RuntimeError( 'swap device %s is larger (%d) than expected (%d)' % (device, actual_size_bytes, safety_limit_bytes)) uuid = get_swap_uuid(device) # wipe FileUtilities.wipe_contents(device, truncate=False) # reinitialize # TRANSLATORS: The variable is a device like /dev/sda2 logger.debug(_("Reinitializing the swap device %s."), device) args = ['mkswap', device] if uuid: args.append("-U") args.append(uuid) (rc, _stdout, stderr) = General.run_external(args) if 0 != rc: raise RuntimeError(stderr.replace("\n", "")) def wipe_memory(): """Wipe unallocated memory""" # cache the file because 'swapoff' changes it proc_swaps = get_proc_swaps() devices = disable_swap_linux() yield True # process GTK+ idle loop # TRANSLATORS: The variable is a device like /dev/sda2 logger.debug(_("Detected these swap devices: %s"), str(devices)) wipe_swap_linux(devices, proc_swaps) yield True child_pid = os.fork() if 0 == child_pid: make_self_oom_target_linux() fill_memory_linux() os._exit(0) else: # TRANSLATORS: This is a debugging message that the parent process is waiting for the child process. logger.debug(_("The function wipe_memory() with process ID {pid} is waiting for child process ID {cid}.").format( pid=os.getpid(), cid=child_pid)) rc = os.waitpid(child_pid, 0)[1] if 0 != rc: logger.warning( _("The child memory-wiping process returned code %d."), rc) enable_swap_linux() yield 0 # how much disk space was recovered bleachbit-4.4.2/bleachbit/Log.py0000664000175000017500000000663614144024253015404 0ustar fabiofabio# vim: ts=4:sw=4:expandtab # BleachBit # Copyright (C) 2008-2021 Andrew Ziem # https://www.bleachbit.org # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . """ Logging """ import logging def is_debugging_enabled_via_cli(): """Return boolean whether user required debugging on the command line""" import sys return any(arg.startswith('--debug') for arg in sys.argv) class DelayLog(object): def __init__(self): self.queue = [] self.msg = '' def read(self): yield from self.queue self.queue = [] def write(self, msg): self.msg += msg if self.msg[-1] == '\n': self.queue.append(self.msg) self.msg = '' def init_log(): """Set up the root logger This is one of the first steps in __init___ """ logger = logging.getLogger('bleachbit') import sys # On Microsoft Windows when running frozen without the console, # avoid py2exe redirecting stderr to bleachbit.exe.log by not # writing to stderr because py2exe redirects stderr to a file. # # sys.frozen = 'console_exe' means the console is shown, which # does not require special handling. if hasattr(sys, 'frozen') and sys.frozen == 'windows_exe': sys.stderr = DelayLog() # debug if command line asks for it or if this a non-final release if is_debugging_enabled_via_cli(): logger.setLevel(logging.DEBUG) else: logger.setLevel(logging.INFO) logger_sh = logging.StreamHandler() logger.addHandler(logger_sh) return logger def set_root_log_level(): """Adjust the root log level This runs later in the application's startup process when the configuration is loaded or after a change via the GUI. """ from bleachbit.Options import options is_debug = options.get('debug') root_logger = logging.getLogger('bleachbit') root_logger.setLevel(logging.DEBUG if is_debug else logging.INFO) class GtkLoggerHandler(logging.Handler): def __init__(self, append_text): logging.Handler.__init__(self) self.append_text = append_text self.msg = '' self.update_log_level() def update_log_level(self): """Set the log level""" from bleachbit.Options import options if options.get('debug'): self.min_level = logging.DEBUG else: self.min_level = logging.WARNING def emit(self, record): if record.levelno < self.min_level: return tag = 'error' if record.levelno >= logging.WARNING else None msg = record.getMessage() if record.exc_text: msg = msg + '\n' + record.exc_text self.append_text(msg + '\n', tag) def write(self, msg): self.msg += msg if self.msg[-1] == '\n': tag = None self.append_text(msg, tag) self.msg = '' bleachbit-4.4.2/bleachbit/Cleaner.py0000775000175000017500000007464314144024253016242 0ustar fabiofabio# vim: ts=4:sw=4:expandtab # BleachBit # Copyright (C) 2008-2021 Andrew Ziem # https://www.bleachbit.org # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . """ Perform (or assist with) cleaning operations. """ import glob import logging import os.path import re import sys from bleachbit import _ from bleachbit.FileUtilities import children_in_directory from bleachbit.Options import options from bleachbit import Command, FileUtilities, Memory, Special # Suppress GTK warning messages while running in CLI #34 import warnings warnings.simplefilter("ignore", Warning) try: from bleachbit.GuiBasic import Gtk, Gdk HAVE_GTK = Gdk.get_default_root_window() is not None except (ImportError, RuntimeError, ValueError) as e: # ImportError happens when GTK is not installed. # RuntimeError can happen when X is not available (e.g., cron, ssh). # ValueError seen on BleachBit 3.0 with GTK 3 (GitHub issue 685) HAVE_GTK = False if 'posix' == os.name: from bleachbit import Unix elif 'nt' == os.name: from bleachbit import Windows # a module-level variable for holding cleaners backends = {} class Cleaner: """Base class for a cleaner""" def __init__(self): self.actions = [] self.id = None self.description = None self.name = None self.options = {} self.running = [] self.warnings = {} self.regexes_compiled = [] def add_action(self, option_id, action): """Register 'action' (instance of class Action) to be executed for ''option_id'. The actions must implement list_files and other_cleanup()""" self.actions += ((option_id, action), ) def add_option(self, option_id, name, description): """Register option (such as 'cache')""" self.options[option_id] = (name, description) def add_running(self, detection_type, pathname): """Add a way to detect this program is currently running""" self.running += ((detection_type, pathname), ) def auto_hide(self): """Return boolean whether it is OK to automatically hide this cleaner""" for (option_id, __name) in self.get_options(): try: for cmd in self.get_commands(option_id): for _dummy in cmd.execute(False): return False for _ds in self.get_deep_scan(option_id): return False except Exception: logger = logging.getLogger(__name__) logger.exception('exception in auto_hide(), cleaner=%s, option=%s', self.name, option_id) return True def get_commands(self, option_id): """Get list of Command instances for option 'option_id'""" for action in self.actions: if option_id == action[0]: yield from action[1].get_commands() if option_id not in self.options: raise RuntimeError("Unknown option '%s'" % option_id) def get_deep_scan(self, option_id): """Get dictionary used to build a deep scan""" for action in self.actions: if option_id == action[0]: try: yield from action[1].get_deep_scan() except StopIteration: return if option_id not in self.options: raise RuntimeError("Unknown option '%s'" % option_id) def get_description(self): """Brief description of the cleaner""" return self.description def get_id(self): """Return the unique name of this cleaner""" return self.id def get_name(self): """Return the human name of this cleaner""" return self.name def get_option_descriptions(self): """Yield the names and descriptions of each option in a 2-tuple""" if self.options: for key in sorted(self.options.keys()): yield (self.options[key][0], self.options[key][1]) def get_options(self): """Return user-configurable options in 2-tuple (id, name)""" if self.options: for key in sorted(self.options.keys()): yield (key, self.options[key][0]) def get_warning(self, option_id): """Return a warning as string.""" if option_id in self.warnings: return self.warnings[option_id] else: return None def is_running(self): """Return whether the program is currently running""" logger = logging.getLogger(__name__) for running in self.running: test = running[0] pathname = running[1] if 'exe' == test and 'posix' == os.name: if Unix.is_running(pathname): logger.debug("process '%s' is running", pathname) return True elif 'exe' == test and 'nt' == os.name: if Windows.is_process_running(pathname): logger.debug("process '%s' is running", pathname) return True elif 'pathname' == test: expanded = os.path.expanduser(os.path.expandvars(pathname)) for globbed in glob.iglob(expanded): if os.path.exists(globbed): logger.debug( "file '%s' exists indicating '%s' is running", globbed, self.name) return True else: raise RuntimeError( "Unknown running-detection test '%s'" % test) return False def is_usable(self): """Return whether the cleaner is usable (has actions)""" return len(self.actions) > 0 def set_warning(self, option_id, description): """Set a warning to be displayed when option is selected interactively""" self.warnings[option_id] = description class OpenOfficeOrg(Cleaner): """Delete OpenOffice.org cache""" def __init__(self): Cleaner.__init__(self) self.options = {} self.add_option('cache', _('Cache'), _('Delete the cache')) self.add_option('recent_documents', _('Most recently used'), _( "Delete the list of recently used documents")) self.id = 'openofficeorg' self.name = 'OpenOffice.org' self.description = _("Office suite") # reference: http://katana.oooninja.com/w/editions_of_openoffice.org if 'posix' == os.name: self.prefixes = ["~/.ooo-2.0", "~/.openoffice.org2", "~/.openoffice.org2.0", "~/.openoffice.org/3"] self.prefixes += ["~/.ooo-dev3"] if 'nt' == os.name: self.prefixes = [ "$APPDATA\\OpenOffice.org\\3", "$APPDATA\\OpenOffice.org2"] def get_commands(self, option_id): # paths for which to run expand_glob_join egj = [] if 'recent_documents' == option_id: egj.append( "user/registry/data/org/openoffice/Office/Histories.xcu") egj.append( "user/registry/cache/org.openoffice.Office.Histories.dat") if 'recent_documents' == option_id and not 'cache' == option_id: egj.append("user/registry/cache/org.openoffice.Office.Common.dat") for egj_ in egj: for prefix in self.prefixes: for path in FileUtilities.expand_glob_join(prefix, egj_): if 'nt' == os.name: path = os.path.normpath(path) if os.path.lexists(path): yield Command.Delete(path) if 'cache' == option_id: dirs = [] for prefix in self.prefixes: dirs += FileUtilities.expand_glob_join( prefix, "user/registry/cache/") for dirname in dirs: if 'nt' == os.name: dirname = os.path.normpath(dirname) for filename in children_in_directory(dirname, False): yield Command.Delete(filename) if 'recent_documents' == option_id: for prefix in self.prefixes: for path in FileUtilities.expand_glob_join(prefix, "user/registry/data/org/openoffice/Office/Common.xcu"): if os.path.lexists(path): yield Command.Function(path, Special.delete_ooo_history, _('Delete the usage history')) # ~/.openoffice.org/3/user/registrymodifications.xcu # Apache OpenOffice.org 3.4.1 from openoffice.org on Ubuntu 13.04 # %AppData%\OpenOffice.org\3\user\registrymodifications.xcu # Apache OpenOffice.org 3.4.1 from openoffice.org on Windows XP for path in FileUtilities.expand_glob_join(prefix, "user/registrymodifications.xcu"): if os.path.lexists(path): yield Command.Function(path, Special.delete_office_registrymodifications, _('Delete the usage history')) class System(Cleaner): """Clean the system in general""" def __init__(self): Cleaner.__init__(self) # # options for Linux and BSD # if 'posix' == os.name: # TRANSLATORS: desktop entries are .desktop files in Linux that # make up the application menu (the menu that shows BleachBit, # Firefox, and others. The .desktop files also associate file # types, so clicking on an .html file in Nautilus brings up # Firefox. # More information: # http://standards.freedesktop.org/menu-spec/latest/index.html#introduction self.add_option('desktop_entry', _('Broken desktop files'), _( 'Delete broken application menu entries and file associations')) self.add_option('cache', _('Cache'), _('Delete the cache')) # TRANSLATORS: Localizations are files supporting specific # languages, so applications appear in Spanish, etc. self.add_option('localizations', _('Localizations'), _( 'Delete files for unwanted languages')) self.set_warning( 'localizations', _("Configure this option in the preferences.")) # TRANSLATORS: 'Rotated logs' refers to old system log files. # Linux systems often have a scheduled job to rotate the logs # which means compress all except the newest log and then delete # the oldest log. You could translate this 'old logs.' self.add_option( 'rotated_logs', _('Rotated logs'), _('Delete old system logs')) self.add_option('recent_documents', _('Recent documents list'), _( 'Delete the list of recently used documents')) self.add_option('trash', _('Trash'), _('Empty the trash')) # # options just for Linux # if sys.platform.startswith('linux'): self.add_option('memory', _('Memory'), # TRANSLATORS: 'free' means 'unallocated' _('Wipe the swap and free memory')) self.set_warning( 'memory', _('This option is experimental and may cause system problems.')) # # options just for Microsoft Windows # if 'nt' == os.name: self.add_option('logs', _('Logs'), _('Delete the logs')) self.add_option( 'memory_dump', _('Memory dump'), _('Delete the file')) self.add_option('muicache', 'MUICache', _('Delete the cache')) # TRANSLATORS: Prefetch is Microsoft Windows jargon. self.add_option('prefetch', _('Prefetch'), _('Delete the cache')) self.add_option( 'recycle_bin', _('Recycle bin'), _('Empty the recycle bin')) # TRANSLATORS: 'Update' is a noun, and 'Update uninstallers' is an option to delete # the uninstallers for software updates. self.add_option('updates', _('Update uninstallers'), _( 'Delete uninstallers for Microsoft updates including hotfixes, service packs, and Internet Explorer updates')) # # options for GTK+ # if HAVE_GTK: self.add_option('clipboard', _('Clipboard'), _( 'The desktop environment\'s clipboard used for copy and paste operations')) # # options common to all platforms # # TRANSLATORS: "Custom" is an option allowing the user to specify which # files and folders will be erased. self.add_option('custom', _('Custom'), _( 'Delete user-specified files and folders')) # TRANSLATORS: 'free' means 'unallocated' self.add_option('free_disk_space', _('Free disk space'), # TRANSLATORS: 'free' means 'unallocated' _('Overwrite free disk space to hide deleted files')) self.set_warning('free_disk_space', _('This option is very slow.')) self.add_option( 'tmp', _('Temporary files'), _('Delete the temporary files')) self.description = _("The system in general") self.id = 'system' self.name = _("System") def get_commands(self, option_id): # cache if 'posix' == os.name and 'cache' == option_id: dirname = os.path.expanduser("~/.cache/") for filename in children_in_directory(dirname, True): if not self.whitelisted(filename): yield Command.Delete(filename) # custom if 'custom' == option_id: for (c_type, c_path) in options.get_custom_paths(): if 'file' == c_type: yield Command.Delete(c_path) elif 'folder' == c_type: for path in children_in_directory(c_path, True): yield Command.Delete(path) yield Command.Delete(c_path) else: raise RuntimeError( 'custom folder has invalid type %s' % c_type) # menu menu_dirs = ['~/.local/share/applications', '~/.config/autostart', '~/.gnome/apps/', '~/.gnome2/panel2.d/default/launchers', '~/.gnome2/vfolders/applications/', '~/.kde/share/apps/RecentDocuments/', '~/.kde/share/mimelnk', '~/.kde/share/mimelnk/application/ram.desktop', '~/.kde2/share/mimelnk/application/', '~/.kde2/share/applnk'] if 'posix' == os.name and 'desktop_entry' == option_id: for dirname in menu_dirs: for filename in [fn for fn in children_in_directory(dirname, False) if fn.endswith('.desktop')]: if Unix.is_broken_xdg_desktop(filename): yield Command.Delete(filename) # unwanted locales if 'posix' == os.name and 'localizations' == option_id: for path in Unix.locales.localization_paths(locales_to_keep=options.get_languages()): if os.path.isdir(path): for f in FileUtilities.children_in_directory(path, True): yield Command.Delete(f) yield Command.Delete(path) # Windows logs if 'nt' == os.name and 'logs' == option_id: paths = ( '$ALLUSERSPROFILE\\Application Data\\Microsoft\\Dr Watson\\*.log', '$ALLUSERSPROFILE\\Application Data\\Microsoft\\Dr Watson\\user.dmp', '$LocalAppData\\Microsoft\\Windows\\WER\\ReportArchive\\*\\*', '$LocalAppData\\Microsoft\\Windows\WER\\ReportQueue\\*\\*', '$programdata\\Microsoft\\Windows\\WER\\ReportArchive\\*\\*', '$programdata\\Microsoft\\Windows\\WER\\ReportQueue\\*\\*', '$localappdata\\Microsoft\\Internet Explorer\\brndlog.bak', '$localappdata\\Microsoft\\Internet Explorer\\brndlog.txt', '$windir\\*.log', '$windir\\imsins.BAK', '$windir\\OEWABLog.txt', '$windir\\SchedLgU.txt', '$windir\\ntbtlog.txt', '$windir\\setuplog.txt', '$windir\\REGLOCS.OLD', '$windir\\Debug\\*.log', '$windir\\Debug\\Setup\\UpdSh.log', '$windir\\Debug\\UserMode\\*.log', '$windir\\Debug\\UserMode\\ChkAcc.bak', '$windir\\Debug\\UserMode\\userenv.bak', '$windir\\Microsoft.NET\Framework\*\*.log', '$windir\\pchealth\\helpctr\\Logs\\hcupdate.log', '$windir\\security\\logs\\*.log', '$windir\\security\\logs\\*.old', '$windir\\SoftwareDistribution\\*.log', '$windir\\SoftwareDistribution\\DataStore\\Logs\\*', '$windir\\system32\\TZLog.log', '$windir\\system32\\config\\systemprofile\\Application Data\\Microsoft\\Internet Explorer\\brndlog.bak', '$windir\\system32\\config\\systemprofile\\Application Data\\Microsoft\\Internet Explorer\\brndlog.txt', '$windir\\system32\\LogFiles\\AIT\\AitEventLog.etl.???', '$windir\\system32\\LogFiles\\Firewall\\pfirewall.log*', '$windir\\system32\\LogFiles\\Scm\\SCM.EVM*', '$windir\\system32\\LogFiles\\WMI\\Terminal*.etl', '$windir\\system32\\LogFiles\\WMI\\RTBackup\EtwRT.*etl', '$windir\\system32\\wbem\\Logs\\*.lo_', '$windir\\system32\\wbem\\Logs\\*.log', ) for path in paths: expanded = os.path.expandvars(path) for globbed in glob.iglob(expanded): yield Command.Delete(globbed) # memory if sys.platform.startswith('linux') and 'memory' == option_id: yield Command.Function(None, Memory.wipe_memory, _('Memory')) # memory dump # how to manually create this file # http://www.pctools.com/guides/registry/detail/856/ if 'nt' == os.name and 'memory_dump' == option_id: fname = os.path.expandvars('$windir\\memory.dmp') if os.path.exists(fname): yield Command.Delete(fname) for fname in glob.iglob(os.path.expandvars('$windir\\Minidump\\*.dmp')): yield Command.Delete(fname) # most recently used documents list if 'posix' == os.name and 'recent_documents' == option_id: ru_fn = os.path.expanduser("~/.recently-used") if os.path.lexists(ru_fn): yield Command.Delete(ru_fn) # GNOME 2.26 (as seen on Ubuntu 9.04) will retain the list # in memory if it is simply deleted, so it must be shredded # (or at least truncated). # # GNOME 2.28.1 (Ubuntu 9.10) and 2.30 (10.04) do not re-read # the file after truncation, but do re-read it after # shredding. # # https://bugzilla.gnome.org/show_bug.cgi?id=591404 def gtk_purge_items(): """Purge GTK items""" Gtk.RecentManager().get_default().purge_items() yield 0 xbel_pathnames = [ '~/.recently-used.xbel', '~/.local/share/recently-used.xbel*', '~/snap/*/*/.local/share/recently-used.xbel'] for path1 in xbel_pathnames: for path2 in glob.iglob(os.path.expanduser(path1)): if os.path.lexists(path2): yield Command.Shred(path2) if HAVE_GTK: # Use the Function to skip when in preview mode yield Command.Function(None, gtk_purge_items, _('Recent documents list')) if 'posix' == os.name and 'rotated_logs' == option_id: for path in Unix.rotated_logs(): yield Command.Delete(path) # temporary files if 'posix' == os.name and 'tmp' == option_id: dirnames = ['/tmp', '/var/tmp'] for dirname in dirnames: for path in children_in_directory(dirname, True): is_open = FileUtilities.openfiles.is_open(path) ok = not is_open and os.path.isfile(path) and \ not os.path.islink(path) and \ FileUtilities.ego_owner(path) and \ not self.whitelisted(path) if ok: yield Command.Delete(path) # temporary files if 'nt' == os.name and 'tmp' == option_id: dirnames = [os.path.expandvars(r'%temp%'), os.path.expandvars("%windir%\\temp\\")] # whitelist the folder %TEMP%\Low but not its contents # https://bugs.launchpad.net/bleachbit/+bug/1421726 for dirname in dirnames: low = os.path.join(dirname, 'low').lower() for filename in children_in_directory(dirname, True): if not low == filename.lower(): yield Command.Delete(filename) # trash if 'posix' == os.name and 'trash' == option_id: dirname = os.path.expanduser("~/.Trash") for filename in children_in_directory(dirname, False): yield Command.Delete(filename) # fixme http://www.ramendik.ru/docs/trashspec.html # http://standards.freedesktop.org/basedir-spec/basedir-spec-0.6.html # ~/.local/share/Trash # * GNOME 2.22, Fedora 9 # * KDE 4.1.3, Ubuntu 8.10 dirname = os.path.expanduser("~/.local/share/Trash/files") for filename in children_in_directory(dirname, True): yield Command.Delete(filename) dirname = os.path.expanduser("~/.local/share/Trash/info") for filename in children_in_directory(dirname, True): yield Command.Delete(filename) dirname = os.path.expanduser("~/.local/share/Trash/expunged") # desrt@irc.gimpnet.org tells me that the trash # backend puts files in here temporary, but in some situations # the files are stuck. for filename in children_in_directory(dirname, True): yield Command.Delete(filename) # clipboard if HAVE_GTK and 'clipboard' == option_id: def clear_clipboard(): clipboard = Gtk.Clipboard.get(Gdk.SELECTION_CLIPBOARD) clipboard.set_text(' ', 1) clipboard.clear() return 0 yield Command.Function(None, clear_clipboard, _('Clipboard')) # overwrite free space shred_drives = options.get_list('shred_drives') if 'free_disk_space' == option_id and shred_drives: for pathname in shred_drives: # TRANSLATORS: 'Free' means 'unallocated.' # %s expands to a path such as C:\ or /tmp/ display = _("Overwrite free disk space %s") % pathname def wipe_path_func(): # Yield control to GTK idle because this process # is very slow. Also display progress. yield from FileUtilities.wipe_path(pathname, idle=True) yield 0 yield Command.Function(None, wipe_path_func, display) # MUICache if 'nt' == os.name and 'muicache' == option_id: keys = ( 'HKCU\\Software\\Microsoft\\Windows\\ShellNoRoam\\MUICache', 'HKCU\\Software\\Classes\\Local Settings\\Software\\Microsoft\\Windows\\Shell\\MuiCache') for key in keys: yield Command.Winreg(key, None) # prefetch if 'nt' == os.name and 'prefetch' == option_id: for path in glob.iglob(os.path.expandvars('$windir\\Prefetch\\*.pf')): yield Command.Delete(path) # recycle bin if 'nt' == os.name and 'recycle_bin' == option_id: # This method allows shredding recycled_any = False for path in Windows.get_recycle_bin(): recycled_any = True yield Command.Delete(path) # Windows 10 refreshes the recycle bin icon when the user # opens the recycle bin folder. # This is a hack to refresh the icon. def empty_recycle_bin_func(): import tempfile tmpdir = tempfile.mkdtemp() Windows.move_to_recycle_bin(tmpdir) try: Windows.empty_recycle_bin(None, True) except: logging.getLogger(__name__).info( 'error in empty_recycle_bin()', exc_info=True) yield 0 # Using the Function Command prevents emptying the recycle bin # when in preview mode. if recycled_any: yield Command.Function(None, empty_recycle_bin_func, _('Empty the recycle bin')) # Windows Updates if 'nt' == os.name and 'updates' == option_id: for wu in Windows.delete_updates(): yield wu def init_whitelist(self): """Initialize the whitelist only once for performance""" regexes = [ '^/tmp/.X0-lock$', '^/tmp/.truecrypt_aux_mnt.*/(control|volume)$', '^/tmp/.vbox-[^/]+-ipc/lock$', '^/tmp/.wine-[0-9]+/server-.*/lock$', '^/tmp/gconfd-[^/]+/lock/ior$', '^/tmp/fsa/', # fsarchiver '^/tmp/kde-', '^/tmp/kdesudo-', '^/tmp/ksocket-', '^/tmp/orbit-[^/]+/bonobo-activation-register[a-z0-9-]*.lock$', '^/tmp/orbit-[^/]+/bonobo-activation-server-[a-z0-9-]*ior$', '^/tmp/pulse-[^/]+/pid$', '^/var/tmp/kdecache-', '^' + os.path.expanduser('~/.cache/wallpaper/'), # Flatpak mount point '^' + os.path.expanduser('~/.cache/doc($|/)'), # Clean Firefox cache from Firefox cleaner (LP#1295826) '^' + os.path.expanduser('~/.cache/mozilla/'), # Clean Google Chrome cache from Google Chrome cleaner (LP#656104) '^' + os.path.expanduser('~/.cache/google-chrome/'), '^' + os.path.expanduser('~/.cache/gnome-control-center/'), # Clean Evolution cache from Evolution cleaner (GitHub #249) '^' + os.path.expanduser('~/.cache/evolution/'), # iBus Pinyin # https://bugs.launchpad.net/bleachbit/+bug/1538919 '^' + os.path.expanduser('~/.cache/ibus/'), # Linux Bluetooth daemon obexd directory is typically empty, so be careful # not to delete the empty directory. '^' + os.path.expanduser('~/.cache/obexd($|/)')] for regex in regexes: self.regexes_compiled.append(re.compile(regex)) def whitelisted(self, pathname): """Return boolean whether file is whitelisted""" if os.name == 'nt': # Whitelist is specific to POSIX return False if not self.regexes_compiled: self.init_whitelist() for regex in self.regexes_compiled: if regex.match(pathname) is not None: return True return False def register_cleaners(cb_progress=lambda x: None, cb_done=lambda: None): """Register all known cleaners: system, CleanerML, and Winapp2""" global backends # wipe out any registrations # Because this is a global variable, cannot use backends = {} backends.clear() # initialize "hard coded" (non-CleanerML) backends backends["openofficeorg"] = OpenOfficeOrg() backends["system"] = System() # register CleanerML cleaners from bleachbit import CleanerML cb_progress(_('Loading native cleaners.')) yield from CleanerML.load_cleaners(cb_progress) # register Winapp2.ini cleaners if 'nt' == os.name: cb_progress(_('Importing cleaners from Winapp2.ini.')) from bleachbit import Winapp yield from Winapp.load_cleaners(cb_progress) cb_done() yield False # end the iteration def create_simple_cleaner(paths): """Shred arbitrary files (used in CLI and GUI)""" cleaner = Cleaner() cleaner.add_option(option_id='files', name='', description='') cleaner.name = _("System") # shows up in progress bar from bleachbit import Action class CustomFileAction(Action.ActionProvider): action_key = '__customfileaction' def get_commands(self): for path in paths: if not isinstance(path, (str)): raise RuntimeError( 'expected path as string but got %s' % str(path)) if not os.path.isabs(path): path = os.path.abspath(path) if os.path.isdir(path): for child in children_in_directory(path, True): yield Command.Shred(child) yield Command.Shred(path) provider = CustomFileAction(None) cleaner.add_action('files', provider) return cleaner def create_wipe_cleaner(path): """Wipe free disk space of arbitrary paths (used in GUI)""" cleaner = Cleaner() cleaner.add_option( option_id='free_disk_space', name='', description='') cleaner.name = '' # create a temporary cleaner object display = _("Overwrite free disk space %s") % path def wipe_path_func(): yield from FileUtilities.wipe_path(path, idle=True) yield 0 from bleachbit import Action class CustomWipeAction(Action.ActionProvider): action_key = '__customwipeaction' def get_commands(self): yield Command.Function(None, wipe_path_func, display) provider = CustomWipeAction(None) cleaner.add_action('free_disk_space', provider) return cleaner bleachbit-4.4.2/bleachbit/Revision.py0000664000175000017500000000002514144024253016443 0ustar fabiofabiorevision = "db612a5" bleachbit-4.4.2/bleachbit/Options.py0000664000175000017500000003371514144024253016314 0ustar fabiofabio# vim: ts=4:sw=4:expandtab # BleachBit # Copyright (C) 2008-2021 Andrew Ziem # https://www.bleachbit.org # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . """ Store and retrieve user preferences """ import bleachbit from bleachbit import General from bleachbit import _ from bleachbit.Log import set_root_log_level import logging import os import re logger = logging.getLogger(__name__) if 'nt' == os.name: from win32file import GetLongPathName boolean_keys = ['auto_hide', 'check_beta', 'check_online_updates', 'dark_mode', 'debug', 'delete_confirmation', 'exit_done', 'first_start', 'remember_geometry', 'shred', 'units_iec', 'window_maximized', 'window_fullscreen'] if 'nt' == os.name: boolean_keys.append('update_winapp2') boolean_keys.append('win10_theme') int_keys = ['window_x', 'window_y', 'window_width', 'window_height', ] def path_to_option(pathname): """Change a pathname to a .ini option name (a key)""" # On Windows change to lowercase and use backwards slashes. pathname = os.path.normcase(pathname) # On Windows expand DOS-8.3-style pathnames. if 'nt' == os.name and os.path.exists(pathname): pathname = GetLongPathName(pathname) if ':' == pathname[1]: # ConfigParser treats colons in a special way pathname = pathname[0] + pathname[2:] return pathname def init_configuration(): """Initialize an empty configuration, if necessary""" if not os.path.exists(bleachbit.options_dir): General.makedirs(bleachbit.options_dir) if os.path.lexists(bleachbit.options_file): logger.debug('Deleting configuration: %s ' % bleachbit.options_file) os.remove(bleachbit.options_file) with open(bleachbit.options_file, 'w', encoding='utf-8-sig') as f_ini: f_ini.write('[bleachbit]\n') if os.name == 'nt' and bleachbit.portable_mode: f_ini.write('[Portable]\n') for section in options.config.sections(): options.config.remove_section(section) options.restore() class Options: """Store and retrieve user preferences""" def __init__(self): self.purged = False self.config = bleachbit.RawConfigParser() self.config.optionxform = str # make keys case sensitive for hashpath purging self.config.BOOLEAN_STATES['t'] = True self.config.BOOLEAN_STATES['f'] = False self.restore() def __flush(self): """Write information to disk""" if not self.purged: self.__purge() if not os.path.exists(bleachbit.options_dir): General.makedirs(bleachbit.options_dir) mkfile = not os.path.exists(bleachbit.options_file) with open(bleachbit.options_file, 'w', encoding='utf-8-sig') as _file: try: self.config.write(_file) except IOError as e: from errno import ENOSPC if e.errno == ENOSPC: logger.error( _("Disk was full when writing configuration to file %s"), bleachbit.options_file) else: raise if mkfile and General.sudo_mode(): General.chownself(bleachbit.options_file) def __purge(self): """Clear out obsolete data""" self.purged = True if not self.config.has_section('hashpath'): return for option in self.config.options('hashpath'): pathname = option if 'nt' == os.name and re.search('^[a-z]\\\\', option): # restore colon lost because ConfigParser treats colon special # in keys pathname = pathname[0] + ':' + pathname[1:] exists = False try: exists = os.path.lexists(pathname) except: # this deals with corrupt keys # https://www.bleachbit.org/forum/bleachbit-wont-launch-error-startup logger.error( _("Error checking whether path exists: %s"), pathname) if not exists: # the file does not on exist, so forget it self.config.remove_option('hashpath', option) def __set_default(self, key, value): """Set the default value""" if not self.config.has_option('bleachbit', key): self.set(key, value) def has_option(self, option, section='bleachbit'): """Check if option is set""" return self.config.has_option(section, option) def get(self, option, section='bleachbit'): """Retrieve a general option""" if not 'nt' == os.name and 'update_winapp2' == option: return False if section == 'hashpath' and option[1] == ':': option = option[0] + option[2:] if option in boolean_keys: from bleachbit.Log import is_debugging_enabled_via_cli if section == 'bleachbit' and option == 'debug' and is_debugging_enabled_via_cli(): # command line overrides store configuration return True return self.config.getboolean(section, option) elif option in int_keys: return self.config.getint(section, option) return self.config.get(section, option) def get_hashpath(self, pathname): """Recall the hash for a file""" return self.get(path_to_option(pathname), 'hashpath') def get_language(self, langid): """Retrieve value for whether to preserve the language""" if not self.config.has_option('preserve_languages', langid): return False return self.config.getboolean('preserve_languages', langid) def get_languages(self): """Return a list of all selected languages""" if not self.config.has_section('preserve_languages'): return None return self.config.options('preserve_languages') def get_list(self, option): """Return an option which is a list data type""" section = "list/%s" % option if not self.config.has_section(section): return None values = [ self.config.get(section, option) for option in sorted(self.config.options(section)) ] return values def get_paths(self, section): """Abstracts get_whitelist_paths and get_custom_paths""" if not self.config.has_section(section): return [] myoptions = [] for option in sorted(self.config.options(section)): pos = option.find('_') if -1 == pos: continue myoptions.append(option[0:pos]) values = [] for option in set(myoptions): p_type = self.config.get(section, option + '_type') p_path = self.config.get(section, option + '_path') values.append((p_type, p_path)) return values def get_whitelist_paths(self): """Return the whitelist of paths""" return self.get_paths("whitelist/paths") def get_custom_paths(self): """Return list of custom paths""" return self.get_paths("custom/paths") def get_tree(self, parent, child): """Retrieve an option for the tree view. The child may be None.""" option = parent if child is not None: option += "." + child if not self.config.has_option('tree', option): return False try: return self.config.getboolean('tree', option) except: # in case of corrupt configuration (Launchpad #799130) logger.exception('Error in get_tree()') return False def is_corrupt(self): """Perform a self-check for corruption of the configuration""" # no boolean key must raise an exception for boolean_key in boolean_keys: try: if self.config.has_option('bleachbit', boolean_key): self.config.getboolean('bleachbit', boolean_key) except ValueError: return True # no int key must raise an exception for int_key in int_keys: try: if self.config.has_option('bleachbit', int_key): self.config.getint('bleachbit', int_key) except ValueError: return True return False def restore(self): """Restore saved options from disk""" try: self.config.read(bleachbit.options_file, encoding='utf-8-sig') except: logger.exception("Error reading application's configuration") if not self.config.has_section("bleachbit"): self.config.add_section("bleachbit") if not self.config.has_section("hashpath"): self.config.add_section("hashpath") if not self.config.has_section("list/shred_drives"): from bleachbit.FileUtilities import guess_overwrite_paths try: self.set_list('shred_drives', guess_overwrite_paths()) except: logger.exception( _("Error when setting the default drives to shred.")) # set defaults self.__set_default("auto_hide", True) self.__set_default("check_beta", False) self.__set_default("check_online_updates", True) self.__set_default("dark_mode", True) self.__set_default("debug", False) self.__set_default("delete_confirmation", True) self.__set_default("exit_done", False) self.__set_default("remember_geometry", True) self.__set_default("shred", False) self.__set_default("units_iec", False) self.__set_default("window_fullscreen", False) self.__set_default("window_maximized", False) if 'nt' == os.name: self.__set_default("update_winapp2", False) self.__set_default("win10_theme", False) if not self.config.has_section('preserve_languages'): lang = bleachbit.user_locale pos = lang.find('_') if -1 != pos: lang = lang[0: pos] for _lang in set([lang, 'en']): logger.info(_("Automatically preserving language %s."), _lang) self.set_language(_lang, True) # BleachBit upgrade or first start ever if not self.config.has_option('bleachbit', 'version') or \ self.get('version') != bleachbit.APP_VERSION: self.set('first_start', True) # set version self.set("version", bleachbit.APP_VERSION) def set(self, key, value, section='bleachbit', commit=True): """Set a general option""" self.config.set(section, key, str(value)) if commit: self.__flush() def commit(self): self.__flush() def set_hashpath(self, pathname, hashvalue): """Remember the hash of a path""" self.set(path_to_option(pathname), hashvalue, 'hashpath') def set_list(self, key, values): """Set a value which is a list data type""" section = "list/%s" % key if self.config.has_section(section): self.config.remove_section(section) self.config.add_section(section) for counter, value in enumerate(values): self.config.set(section, str(counter), value) self.__flush() def set_whitelist_paths(self, values): """Save the whitelist""" section = "whitelist/paths" if self.config.has_section(section): self.config.remove_section(section) self.config.add_section(section) for counter, value in enumerate(values): self.config.set(section, str(counter) + '_type', value[0]) self.config.set(section, str(counter) + '_path', value[1]) self.__flush() def set_custom_paths(self, values): """Save the customlist""" section = "custom/paths" if self.config.has_section(section): self.config.remove_section(section) self.config.add_section(section) for counter, value in enumerate(values): self.config.set(section, str(counter) + '_type', value[0]) self.config.set(section, str(counter) + '_path', value[1]) self.__flush() def set_language(self, langid, value): """Set the value for a locale (whether to preserve it)""" if not self.config.has_section('preserve_languages'): self.config.add_section('preserve_languages') if self.config.has_option('preserve_languages', langid) and not value: self.config.remove_option('preserve_languages', langid) else: self.config.set('preserve_languages', langid, str(value)) self.__flush() def set_tree(self, parent, child, value): """Set an option for the tree view. The child may be None.""" if not self.config.has_section("tree"): self.config.add_section("tree") option = parent if child is not None: option = option + "." + child if self.config.has_option('tree', option) and not value: self.config.remove_option('tree', option) else: self.config.set('tree', option, str(value)) self.__flush() def toggle(self, key): """Toggle a boolean key""" self.set(key, not self.get(key)) options = Options() # Now that the configuration is loaded, honor the debug preference there. set_root_log_level() bleachbit-4.4.2/bleachbit/Chaff.py0000664000175000017500000002720214144024253015662 0ustar fabiofabio# vim: ts=4:sw=4:expandtab # -*- coding: UTF-8 -*- # BleachBit # Copyright (C) 2008-2021 Andrew Ziem # https://www.bleachbit.org # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import email.generator from email.mime.text import MIMEText import json import logging import os import random import tempfile from datetime import datetime import queue as _unused_module_Queue from bleachbit import _, bleachbit_exe_path from bleachbit import options_dir from . import markovify logger = logging.getLogger(__name__) # These were typos in the original emails, not OCR errors: # abdinh@state.gov # mhcaleja@state.gov RECIPIENTS = [ 'abedinh@state.gov', 'adlerce@state.gov', 'baerdb@state.gov', 'baldersonkm@state.gov', 'balderstonkm@state.gov', 'bam@mikulski.senate.gov', 'bealeca@state.gov', 'benjamin_moncrief@lemieux.senate.gov', 'blaker2@state.gov', 'brimmere@state.gov', 'burnswj@state.gov', 'butzgych2@state.gov', 'campbellkm@state.gov', 'carsonj@state.gov', 'cholletdh@state.gov', 'cindy.buhl@mail.house.gov', 'colemancl@state.gov', 'crowleypj@state.gov', 'danieljj@state.gov', 'david_garten@lautenberg.senate.gov', 'dewanll@state.gov', 'feltmanjd@state.gov', 'fuchsmh@state.gov', 'goldbergps@state.gov', 'goldenjr@state.gov', 'gonzalezjs@state.gov', 'gordonph@state.gov', 'hanleymr@state.gov', 'hdr22@clintonemail.com', 'hillcr@state.gov', 'holbrookerc@state.gov', 'hormatsrd@state.gov', 'hr15@att.blackberry.net', 'hr15@mycingular.blackberry.net', 'hrod17@clintonemail.com', 'huma@clintonemail.com', 'hyded@state.gov', 'info@mailva.evite.com', 'jilotylc@state.gov', 'jonespw2@state.gov', 'kellyc@state.gov', 'klevorickcb@state.gov', 'kohhh@state.gov', 'laszczychj@state.gov', 'lewjj@state.gov', 'macmanusje@state.gov', 'marshallcp@state.gov', 'mchaleja@state.gov', 'millscd@state.gov', 'millscd@state.gov', 'muscatinel@state.gov', 'nidestr@state.gov', 'nulandvj@state.gov', 'oterom2@state.gov', 'posnermh@state.gov', 'reinesp@state.gov', 'reinespi@state.gov', 'ricese@state.gov', 'rodriguezme@state.gov', 'rooneym@state.gov', 's_specialassistants@state.gov', 'schwerindb@state.gov', 'shannonta@state.gov', 'shapiroa@state.gov', 'shermanwr@state.gov', 'slaughtera@state.gov', 'steinbergjb@state.gov', 'sterntd@state.gov', 'sullivanjj@state.gov', 'tauschereo@state.gov', 'tillemannts@state.gov', 'toivnf@state.gov', 'tommy_ross@reid.senate.gov', 'valenzuelaaa@state.gov', 'valmorolj@state.gov', 'valmorolj@state.gov', 'vermarr@state.gov', 'verveerms@state.gov', 'woodardew@state.gov'] DEFAULT_SUBJECT_LENGTH = 64 DEFAULT_NUMBER_OF_SENTENCES_CLINTON = 50 DEFAULT_NUMBER_OF_SENTENCES_2600 = 50 MODEL_BASENAMES = ( '2600_model.json.bz2', 'clinton_content_model.json.bz2', 'clinton_subject_model.json.bz2') URL_TEMPLATES = ( 'https://sourceforge.net/projects/bleachbit/files/chaff/%s/download', 'https://download.bleachbit.org/chaff/%s') DEFAULT_MODELS_DIR = options_dir def _load_model(model_path): _open = open if model_path.endswith('.bz2'): import bz2 _open = bz2.open with _open(model_path, 'rt', encoding='utf-8') as model_file: return markovify.Text.from_dict(json.load(model_file)) def load_subject_model(model_path): return _load_model(model_path) def load_content_model(model_path): return _load_model(model_path) def load_2600_model(model_path): return _load_model(model_path) def _get_random_recipient(): return random.choice(RECIPIENTS) def _get_random_datetime(min_year=2011, max_year=2012): date = datetime.strptime('{} {}'.format(random.randint( 1, 365), random.randint(min_year, max_year)), '%j %Y') # Saturday, September 15, 2012 2:20 PM return date.strftime('%A, %B %d, %Y %I:%M %p') def _get_random_content(content_model, number_of_sentences=DEFAULT_NUMBER_OF_SENTENCES_CLINTON): content = [] for _ in range(number_of_sentences): content.append(content_model.make_sentence()) content.append(random.choice([' ', ' ', '\n\n'])) try: return MIMEText(''.join(content), _charset='iso-8859-1') except UnicodeEncodeError: return _get_random_content(content_model, number_of_sentences=number_of_sentences) def _generate_email(subject_model, content_model, number_of_sentences=DEFAULT_NUMBER_OF_SENTENCES_CLINTON, subject_length=DEFAULT_SUBJECT_LENGTH): message = _get_random_content( content_model, number_of_sentences=number_of_sentences) message['Subject'] = subject_model.make_short_sentence(subject_length) message['To'] = _get_random_recipient() message['From'] = _get_random_recipient() message['Sent'] = _get_random_datetime() return message def download_url_to_fn(url, fn, on_error=None, max_retries=2, backoff_factor=0.5): """Download a URL to the given filename""" logger.info('Downloading %s to %s', url, fn) import requests import sys if hasattr(sys, 'frozen'): # when frozen by py2exe, certificates are in alternate location CA_BUNDLE = os.path.join(bleachbit_exe_path, 'cacert.pem') requests.utils.DEFAULT_CA_BUNDLE_PATH = CA_BUNDLE requests.adapters.DEFAULT_CA_BUNDLE_PATH = CA_BUNDLE from urllib3.util.retry import Retry from requests.adapters import HTTPAdapter # 408: request timeout # 429: too many requests # 500: internal server error # 502: bad gateway # 503: service unavailable # 504: gateway_timeout status_forcelist = (408, 429, 500, 502, 503, 504) # sourceforge.net directories to download mirror retries = Retry(total=max_retries, backoff_factor=backoff_factor, status_forcelist=status_forcelist, redirect=5) msg = _('Downloading url failed: %s') % url from bleachbit.Update import user_agent headers = {'user_agent': user_agent()} def do_error(msg2): if on_error: on_error(msg, msg2) from bleachbit.FileUtilities import delete delete(fn, ignore_missing=True) # delete any partial download with requests.Session() as session: session.mount('http://', HTTPAdapter(max_retries=retries)) try: response = session.get(url, headers=headers) content = response.content except requests.exceptions.RequestException as exc: msg2 = '{}: {}'.format(type(exc).__name__, exc) logger.exception(msg) do_error(msg2) return False else: if not response.status_code == 200: logger.error(msg) msg2 = 'Status code: %s' % response.status_code do_error(msg2) return False with open(fn, 'wb') as f: f.write(content) return True def download_models(models_dir=DEFAULT_MODELS_DIR, on_error=None): """Download models Calls on_error(primary_message, secondary_message) in case of error Returns success as boolean value """ for basename in (MODEL_BASENAMES): fn = os.path.join(models_dir, basename) if os.path.exists(fn): logger.debug('File %s already exists', fn) continue this_file_success = False for url_template in URL_TEMPLATES: url = url_template % basename if download_url_to_fn(url, fn, on_error): this_file_success = True break if not this_file_success: return False return True def generate_emails(number_of_emails, email_output_dir, models_dir=DEFAULT_MODELS_DIR, number_of_sentences=DEFAULT_NUMBER_OF_SENTENCES_CLINTON, on_progress=None, *kwargs): logger.debug('Loading two email models') subject_model_path = os.path.join( models_dir, 'clinton_subject_model.json.bz2') content_model_path = os.path.join( models_dir, 'clinton_content_model.json.bz2') subject_model = load_subject_model(subject_model_path) content_model = load_content_model(content_model_path) logger.debug('Generating {:,} emails'.format(number_of_emails)) generated_file_names = [] for i in range(1, number_of_emails + 1): with tempfile.NamedTemporaryFile(mode='w+', prefix='outlook-', suffix='.eml', dir=email_output_dir, delete=False) as email_output_file: email_generator = email.generator.Generator(email_output_file) msg = _generate_email( subject_model, content_model, number_of_sentences=number_of_sentences) email_generator.write(msg.as_string()) generated_file_names.append(email_output_file.name) if on_progress: on_progress(1.0*i/number_of_emails) return generated_file_names def _generate_2600_file(model, number_of_sentences=DEFAULT_NUMBER_OF_SENTENCES_2600): content = [] for _ in range(number_of_sentences): content.append(model.make_sentence()) # The space is repeated to make paragraphs longer. content.append(random.choice([' ', ' ', '\n\n'])) return ''.join(content) def generate_2600(file_count, output_dir, model_dir=DEFAULT_MODELS_DIR, on_progress=None): logger.debug('Loading 2600 model') model_path = os.path.join(model_dir, '2600_model.json.bz2') model = _load_model(model_path) logger.debug('Generating {:,} files'.format(file_count)) generated_file_names = [] for i in range(1, file_count + 1): with tempfile.NamedTemporaryFile(mode='w+', encoding='utf-8', prefix='2600-', suffix='.txt', dir=output_dir, delete=False) as output_file: txt = _generate_2600_file(model) output_file.write(txt) generated_file_names.append(output_file.name) if on_progress: on_progress(1.0*i/file_count) return generated_file_names def have_models(): """Check whether the models exist in the default location. Used to check whether download is needed.""" for basename in (MODEL_BASENAMES): fn = os.path.join(DEFAULT_MODELS_DIR, basename) if not os.path.exists(fn): return False return True bleachbit-4.4.2/bleachbit/Unix.py0000775000175000017500000006407114144024253015606 0ustar fabiofabio# vim: ts=4:sw=4:expandtab # -*- coding: UTF-8 -*- # BleachBit # Copyright (C) 2008-2021 Andrew Ziem # https://www.bleachbit.org # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . """ Integration specific to Unix-like operating systems """ import bleachbit from bleachbit import FileUtilities, General from bleachbit import _ import glob import logging import os import re import shlex import subprocess import sys logger = logging.getLogger(__name__) try: Pattern = re.Pattern except AttributeError: Pattern = re._pattern_type JOURNALD_REGEX = r'^Vacuuming done, freed ([\d.]+[BKMGT]?) of archived journals (on disk|from [\w/]+).$' class LocaleCleanerPath: """This represents a path with either a specific folder name or a folder name pattern. It also may contain several compiled regex patterns for localization items (folders or files) and additional LocaleCleanerPaths that get traversed when asked to supply a list of localization items""" def __init__(self, location): if location is None: raise RuntimeError("location is none") self.pattern = location self.children = [] def add_child(self, child): """Adds a child LocaleCleanerPath""" self.children.append(child) return child def add_path_filter(self, pre, post): """Adds a filter consisting of a prefix and a postfix (e.g. 'foobar_' and '\.qm' to match 'foobar_en_US.utf-8.qm)""" try: regex = re.compile('^' + pre + Locales.localepattern + post + '$') except Exception as errormsg: raise RuntimeError( "Malformed regex '%s' or '%s': %s" % (pre, post, errormsg)) self.add_child(regex) def get_subpaths(self, basepath): """Returns direct subpaths for this object, i.e. either the named subfolder or all subfolders matching the pattern""" if isinstance(self.pattern, Pattern): return (os.path.join(basepath, p) for p in os.listdir(basepath) if self.pattern.match(p) and os.path.isdir(os.path.join(basepath, p))) path = os.path.join(basepath, self.pattern) return [path] if os.path.isdir(path) else [] def get_localizations(self, basepath): """Returns all localization items for this object and all descendant objects""" for path in self.get_subpaths(basepath): for child in self.children: if isinstance(child, LocaleCleanerPath): yield from child.get_localizations(path) elif isinstance(child, Pattern): for element in os.listdir(path): match = child.match(element) if match is not None: yield (match.group('locale'), match.group('specifier'), os.path.join(path, element)) class Locales: """Find languages and localization files""" # The regular expression to match locale strings and extract the langcode. # See test_locale_regex() in tests/TestUnix.py for examples # This doesn't match all possible valid locale strings to avoid # matching filenames you might want to keep, e.g. the regex # to match jp.eucJP might also match jp.importantfileextension localepattern =\ r'(?P[a-z]{2,3})' \ r'(?P[_-][A-Z]{2,4})?(?:\.[\w]+[\d-]+|@\w+)?' \ r'(?P[.-_](?:(?:ISO|iso|UTF|utf|us-ascii)[\d-]+|(?:euc|EUC)[A-Z]+))?' native_locale_names = \ {'aa': 'Afaraf', 'ab': 'аҧсуа бызшәа', 'ace': 'بهسا اچيه', 'ach': 'Acoli', 'ae': 'avesta', 'af': 'Afrikaans', 'ak': 'Akan', 'am': 'አማርኛ', 'an': 'aragonés', 'ang': 'Old English', 'anp': 'Angika', 'ar': 'العربية', 'as': 'অসমীয়া', 'ast': 'Asturianu', 'av': 'авар мацӀ', 'ay': 'aymar aru', 'az': 'azərbaycan dili', 'ba': 'башҡорт теле', 'bal': 'Baluchi', 'be': 'Беларуская мова', 'bg': 'български език', 'bh': 'भोजपुरी', 'bi': 'Bislama', 'bm': 'bamanankan', 'bn': 'বাংলা', 'bo': 'བོད་ཡིག', 'br': 'brezhoneg', 'brx': 'Bodo (India)', 'bs': 'босански', 'byn': 'Bilin', 'ca': 'català', 'ce': 'нохчийн мотт', 'cgg': 'Chiga', 'ch': 'Chamoru', 'ckb': 'Central Kurdish', 'co': 'corsu', 'cr': 'ᓀᐦᐃᔭᐍᐏᐣ', 'crh': 'Crimean Tatar', 'cs': 'česky', 'csb': 'Cashubian', 'cu': 'ѩзыкъ словѣньскъ', 'cv': 'чӑваш чӗлхи', 'cy': 'Cymraeg', 'da': 'dansk', 'de': 'Deutsch', 'doi': 'डोगरी; ڈوگرى', 'dv': 'ދިވެހި', 'dz': 'རྫོང་ཁ', 'ee': 'Eʋegbe', 'el': 'Ελληνικά', 'en': 'English', 'en_AU': 'Australian English', 'en_CA': 'Canadian English', 'en_GB': 'British English', 'eo': 'Esperanto', 'es': 'Español', 'es_419': 'Latin American Spanish', 'et': 'eesti', 'eu': 'euskara', 'fa': 'فارسی', 'ff': 'Fulfulde', 'fi': 'suomen kieli', 'fil': 'Wikang Filipino', 'fin': 'suomen kieli', 'fj': 'vosa Vakaviti', 'fo': 'føroyskt', 'fr': 'Français', 'frp': 'Arpitan', 'fur': 'Frilian', 'fy': 'Frysk', 'ga': 'Gaeilge', 'gd': 'Gàidhlig', 'gez': 'Geez', 'gl': 'galego', 'gn': 'Avañeẽ', 'gu': 'Gujarati', 'gv': 'Gaelg', 'ha': 'هَوُسَ', 'haw': 'Hawaiian', 'he': 'עברית', 'hi': 'हिन्दी', 'hne': 'Chhattisgarhi', 'ho': 'Hiri Motu', 'hr': 'Hrvatski', 'hsb': 'Upper Sorbian', 'ht': 'Kreyòl ayisyen', 'hu': 'Magyar', 'hy': 'Հայերեն', 'hz': 'Otjiherero', 'ia': 'Interlingua', 'id': 'Indonesian', 'ie': 'Interlingue', 'ig': 'Asụsụ Igbo', 'ii': 'ꆈꌠ꒿', 'ik': 'Iñupiaq', 'ilo': 'Ilokano', 'ina': 'Interlingua', 'io': 'Ido', 'is': 'Íslenska', 'it': 'Italiano', 'iu': 'ᐃᓄᒃᑎᑐᑦ', 'iw': 'עברית', 'ja': '日本語', 'jv': 'basa Jawa', 'ka': 'ქართული', 'kab': 'Tazwawt', 'kac': 'Jingpho', 'kg': 'Kikongo', 'ki': 'Gĩkũyũ', 'kj': 'Kuanyama', 'kk': 'қазақ тілі', 'kl': 'kalaallisut', 'km': 'ខ្មែរ', 'kn': 'ಕನ್ನಡ', 'ko': '한국어', 'kok': 'Konkani', 'kr': 'Kanuri', 'ks': 'कश्मीरी', 'ku': 'Kurdî', 'kv': 'коми кыв', 'kw': 'Kernewek', 'ky': 'Кыргызча', 'la': 'latine', 'lb': 'Lëtzebuergesch', 'lg': 'Luganda', 'li': 'Limburgs', 'ln': 'Lingála', 'lo': 'ພາສາລາວ', 'lt': 'lietuvių kalba', 'lu': 'Tshiluba', 'lv': 'latviešu valoda', 'mai': 'Maithili', 'mg': 'fiteny malagasy', 'mh': 'Kajin M̧ajeļ', 'mhr': 'Eastern Mari', 'mi': 'te reo Māori', 'mk': 'македонски јазик', 'ml': 'മലയാളം', 'mn': 'монгол', 'mni': 'Manipuri', 'mr': 'मराठी', 'ms': 'بهاس ملايو', 'mt': 'Malti', 'my': 'ဗမာစာ', 'na': 'Ekakairũ Naoero', 'nb': 'Bokmål', 'nd': 'isiNdebele', 'nds': 'Plattdüütsch', 'ne': 'नेपाली', 'ng': 'Owambo', 'nl': 'Nederlands', 'nn': 'Norsk nynorsk', 'no': 'Norsk', 'nr': 'isiNdebele', 'nso': 'Pedi', 'nv': 'Diné bizaad', 'ny': 'chiCheŵa', 'oc': 'occitan', 'oj': 'ᐊᓂᔑᓈᐯᒧᐎᓐ', 'om': 'Afaan Oromoo', 'or': 'ଓଡ଼ିଆ', 'os': 'ирон æвзаг', 'pa': 'ਪੰਜਾਬੀ', 'pap': 'Papiamentu', 'pau': 'a tekoi er a Belau', 'pi': 'पाऴि', 'pl': 'polski', 'ps': 'پښتو', 'pt': 'Português', 'pt_BR': 'Português do Brasil', 'qu': 'Runa Simi', 'rm': 'rumantsch grischun', 'rn': 'Ikirundi', 'ro': 'română', 'ru': 'Pусский', 'rw': 'Ikinyarwanda', 'sa': 'संस्कृतम्', 'sat': 'ᱥᱟᱱᱛᱟᱲᱤ', 'sc': 'sardu', 'sd': 'सिन्धी', 'se': 'Davvisámegiella', 'sg': 'yângâ tî sängö', 'shn': 'Shan', 'si': 'සිංහල', 'sk': 'slovenčina', 'sl': 'slovenščina', 'sm': 'gagana faa Samoa', 'sn': 'chiShona', 'so': 'Soomaaliga', 'sq': 'Shqip', 'sr': 'Српски', 'ss': 'SiSwati', 'st': 'Sesotho', 'su': 'Basa Sunda', 'sv': 'svenska', 'sw': 'Kiswahili', 'ta': 'தமிழ்', 'te': 'తెలుగు', 'tet': 'Tetum', 'tg': 'тоҷикӣ', 'th': 'ไทย', 'ti': 'ትግርኛ', 'tig': 'Tigre', 'tk': 'Türkmen', 'tl': 'ᜏᜒᜃᜅ᜔ ᜆᜄᜎᜓᜄ᜔', 'tn': 'Setswana', 'to': 'faka Tonga', 'tr': 'Türkçe', 'ts': 'Xitsonga', 'tt': 'татар теле', 'tw': 'Twi', 'ty': 'Reo Tahiti', 'ug': 'Uyghur', 'uk': 'Українська', 'ur': 'اردو', 'uz': 'Ўзбек', 've': 'Tshivenḓa', 'vi': 'Tiếng Việt', 'vo': 'Volapük', 'wa': 'walon', 'wae': 'Walser', 'wal': 'Wolaytta', 'wo': 'Wollof', 'xh': 'isiXhosa', 'yi': 'ייִדיש', 'yo': 'Yorùbá', 'za': 'Saɯ cueŋƅ', 'zh': '中文', 'zh_CN': '中文', 'zh_TW': '中文', 'zu': 'isiZulu'} def __init__(self): self._paths = LocaleCleanerPath(location='/') def add_xml(self, xml_node, parent=None): """Parses the xml data and adds nodes to the LocaleCleanerPath-tree""" if parent is None: parent = self._paths if xml_node.ELEMENT_NODE != xml_node.nodeType: return # if a pattern is supplied, we recurse into all matching subdirectories if 'regexfilter' == xml_node.nodeName: pre = xml_node.getAttribute('prefix') or '' post = xml_node.getAttribute('postfix') or '' parent.add_path_filter(pre, post) elif 'path' == xml_node.nodeName: if xml_node.hasAttribute('directoryregex'): pattern = xml_node.getAttribute('directoryregex') if '/' in pattern: raise RuntimeError( 'directoryregex may not contain slashes.') pattern = re.compile(pattern) parent = parent.add_child(LocaleCleanerPath(pattern)) # a combination of directoryregex and filter could be too much else: if xml_node.hasAttribute("location"): # if there's a filter attribute, it should apply to this path parent = parent.add_child(LocaleCleanerPath( xml_node.getAttribute('location'))) if xml_node.hasAttribute('filter'): userfilter = xml_node.getAttribute('filter') if 1 != userfilter.count('*'): raise RuntimeError( "Filter string '%s' must contain the placeholder * exactly once" % userfilter) # we can't use re.escape, because it escapes too much (pre, post) = (re.sub(r'([\[\]()^$.])', r'\\\1', p) for p in userfilter.split('*')) parent.add_path_filter(pre, post) else: raise RuntimeError( "Invalid node '%s', expected '' or ''" % xml_node.nodeName) # handle child nodes for child_xml in xml_node.childNodes: self.add_xml(child_xml, parent) def localization_paths(self, locales_to_keep): """Returns all localization items matching the previously added xml configuration""" if not locales_to_keep: raise RuntimeError('Found no locales to keep') purgeable_locales = frozenset((locale for locale in Locales.native_locale_names.keys() if locale not in locales_to_keep)) for (locale, specifier, path) in self._paths.get_localizations('/'): specific = locale + (specifier or '') if specific in purgeable_locales or \ (locale in purgeable_locales and specific not in locales_to_keep): yield path def __is_broken_xdg_desktop_application(config, desktop_pathname): """Returns boolean whether application desktop entry file is broken""" if not config.has_option('Desktop Entry', 'Exec'): logger.info( "is_broken_xdg_menu: missing required option 'Exec': '%s'", desktop_pathname) return True exe = config.get('Desktop Entry', 'Exec').split(" ")[0] if not FileUtilities.exe_exists(exe): logger.info( "is_broken_xdg_menu: executable '%s' does not exist '%s'", exe, desktop_pathname) return True if 'env' == exe: # Wine v1.0 creates .desktop files like this # Exec=env WINEPREFIX="/home/z/.wine" wine "C:\\Program # Files\\foo\\foo.exe" execs = shlex.split(config.get('Desktop Entry', 'Exec')) wineprefix = None del execs[0] while True: if execs[0].find("=") < 0: break (name, value) = execs[0].split("=") if name == 'WINEPREFIX': wineprefix = value del execs[0] if not FileUtilities.exe_exists(execs[0]): logger.info( "is_broken_xdg_menu: executable '%s' does not exist '%s'", execs[0], desktop_pathname) return True # check the Windows executable exists if wineprefix: windows_exe = wine_to_linux_path(wineprefix, execs[1]) if not os.path.exists(windows_exe): logger.info("is_broken_xdg_menu: Windows executable '%s' does not exist '%s'", windows_exe, desktop_pathname) return True return False def is_unregistered_mime(mimetype): """Returns True if the MIME type is known to be unregistered. If registered or unknown, conservatively returns False.""" try: from gi.repository import Gio if 0 == len(Gio.app_info_get_all_for_type(mimetype)): return True except ImportError: logger.warning( 'error calling gio.app_info_get_all_for_type(%s)', mimetype) return False def is_broken_xdg_desktop(pathname): """Returns boolean whether the given XDG desktop entry file is broken. Reference: http://standards.freedesktop.org/desktop-entry-spec/latest/""" config = bleachbit.RawConfigParser() config.read(pathname) if not config.has_section('Desktop Entry'): logger.info( "is_broken_xdg_menu: missing required section 'Desktop Entry': '%s'", pathname) return True if not config.has_option('Desktop Entry', 'Type'): logger.info( "is_broken_xdg_menu: missing required option 'Type': '%s'", pathname) return True file_type = config.get('Desktop Entry', 'Type').strip().lower() if 'link' == file_type: if not config.has_option('Desktop Entry', 'URL') and \ not config.has_option('Desktop Entry', 'URL[$e]'): logger.info( "is_broken_xdg_menu: missing required option 'URL': '%s'", pathname) return True return False if 'mimetype' == file_type: if not config.has_option('Desktop Entry', 'MimeType'): logger.info( "is_broken_xdg_menu: missing required option 'MimeType': '%s'", pathname) return True mimetype = config.get('Desktop Entry', 'MimeType').strip().lower() if is_unregistered_mime(mimetype): logger.info( "is_broken_xdg_menu: MimeType '%s' not registered '%s'", mimetype, pathname) return True return False if 'application' != file_type: logger.warning("unhandled type '%s': file '%s'", file_type, pathname) return False if __is_broken_xdg_desktop_application(config, pathname): return True return False def is_running_darwin(exename): try: ps_out = subprocess.check_output(["ps", "aux", "-c"], universal_newlines=True) processess = (re.split(r"\s+", p, 10)[10] for p in ps_out.split("\n") if p != "") next(processess) # drop the header return exename in processess except IndexError: raise RuntimeError("Unexpected output from ps") def is_running_linux(exename): """Check whether exename is running""" for filename in glob.iglob("/proc/*/exe"): try: target = os.path.realpath(filename) except TypeError: # happens, for example, when link points to # '/etc/password\x00 (deleted)' continue except OSError: # 13 = permission denied continue # Google Chrome shows 74 on Ubuntu 19.04 shows up as # /opt/google/chrome/chrome (deleted) found_exename = os.path.basename(target).replace(' (deleted)', '') if exename == found_exename: return True return False def is_running(exename): """Check whether exename is running""" if sys.platform.startswith('linux'): return is_running_linux(exename) elif ('darwin' == sys.platform or sys.platform.startswith('openbsd') or sys.platform.startswith('freebsd')): return is_running_darwin(exename) else: raise RuntimeError('unsupported platform for physical_free()') def rotated_logs(): """Yield a list of rotated (i.e., old) logs in /var/log/""" # Ubuntu 9.04 # /var/log/dmesg.0 # /var/log/dmesg.1.gz # Fedora 10 # /var/log/messages-20090118 globpaths = ('/var/log/*.[0-9]', '/var/log/*/*.[0-9]', '/var/log/*.gz', '/var/log/*/*gz', '/var/log/*/*.old', '/var/log/*.old') for globpath in globpaths: yield from glob.iglob(globpath) regex = '-[0-9]{8}$' globpaths = ('/var/log/*-*', '/var/log/*/*-*') for path in FileUtilities.globex(globpaths, regex): whitelist_re = '^/var/log/(removed_)?(packages|scripts)' if re.match(whitelist_re, path) is None: # for Slackware, Launchpad #367575 yield path def wine_to_linux_path(wineprefix, windows_pathname): """Return a Linux pathname from an absolute Windows pathname and Wine prefix""" drive_letter = windows_pathname[0] windows_pathname = windows_pathname.replace(drive_letter + ":", "drive_" + drive_letter.lower()) windows_pathname = windows_pathname.replace("\\", "/") return os.path.join(wineprefix, windows_pathname) def run_cleaner_cmd(cmd, args, freed_space_regex=r'[\d.]+[kMGTE]?B?', error_line_regexes=None): """Runs a specified command and returns how much space was (reportedly) freed. The subprocess shouldn't need any user input and the user should have the necessary rights. freed_space_regex gets applied to every output line, if the re matches, add values captured by the single group in the regex""" if not FileUtilities.exe_exists(cmd): raise RuntimeError(_('Executable not found: %s') % cmd) freed_space_regex = re.compile(freed_space_regex) error_line_regexes = [re.compile(regex) for regex in error_line_regexes or []] env = {'LC_ALL': 'C', 'PATH': os.getenv('PATH')} output = subprocess.check_output([cmd] + args, stderr=subprocess.STDOUT, universal_newlines=True, env=env) freed_space = 0 for line in output.split('\n'): m = freed_space_regex.match(line) if m is not None: freed_space += FileUtilities.human_to_bytes(m.group(1)) for error_re in error_line_regexes: if error_re.search(line): raise RuntimeError('Invalid output from %s: %s' % (cmd, line)) return freed_space def journald_clean(): """Clean the system journals""" try: return run_cleaner_cmd('journalctl', ['--vacuum-size=1'], JOURNALD_REGEX) except subprocess.CalledProcessError as e: raise RuntimeError("Error calling '%s':\n%s" % (' '.join(e.cmd), e.output)) def apt_autoremove(): """Run 'apt-get autoremove' and return the size (un-rounded, in bytes) of freed space""" args = ['--yes', 'autoremove'] # After this operation, 74.7MB disk space will be freed. # After this operation, 44.0 kB disk space will be freed. freed_space_regex = r'.*, ([\d.]+ ?[a-zA-Z]{2}) disk space will be freed.' try: return run_cleaner_cmd('apt-get', args, freed_space_regex, ['^E: ']) except subprocess.CalledProcessError as e: raise RuntimeError("Error calling '%s':\n%s" % (' '.join(e.cmd), e.output)) def apt_autoclean(): """Run 'apt-get autoclean' and return the size (un-rounded, in bytes) of freed space""" try: return run_cleaner_cmd('apt-get', ['autoclean'], r'^Del .*\[([\d.]+[a-zA-Z]{2})}]', ['^E: ']) except subprocess.CalledProcessError as e: raise RuntimeError("Error calling '%s':\n%s" % (' '.join(e.cmd), e.output)) def apt_clean(): """Run 'apt-get clean' and return the size in bytes of freed space""" old_size = get_apt_size() try: run_cleaner_cmd('apt-get', ['clean'], '^unused regex$', ['^E: ']) except subprocess.CalledProcessError as e: raise RuntimeError("Error calling '%s':\n%s" % (' '.join(e.cmd), e.output)) new_size = get_apt_size() return old_size - new_size def get_apt_size(): """Return the size of the apt cache (in bytes)""" (rc, stdout, stderr) = General.run_external(['apt-get', '-s', 'clean']) paths = re.findall('/[/a-z\.\*]+', stdout) return get_globs_size(paths) def get_globs_size(paths): """Get the cumulative size (in bytes) of a list of globs""" total_size = 0 for path in paths: for p in glob.iglob(path): total_size += FileUtilities.getsize(p) return total_size def yum_clean(): """Run 'yum clean all' and return size in bytes recovered""" if os.path.exists('/var/run/yum.pid'): msg = _( "%s cannot be cleaned because it is currently running. Close it, and try again.") % "Yum" raise RuntimeError(msg) old_size = FileUtilities.getsizedir('/var/cache/yum') args = ['--enablerepo=*', 'clean', 'all'] invalid = ['You need to be root', 'Cannot remove rpmdb file'] run_cleaner_cmd('yum', args, '^unused regex$', invalid) new_size = FileUtilities.getsizedir('/var/cache/yum') return old_size - new_size def dnf_clean(): """Run 'dnf clean all' and return size in bytes recovered""" if os.path.exists('/var/run/dnf.pid'): msg = _( "%s cannot be cleaned because it is currently running. Close it, and try again.") % "Dnf" raise RuntimeError(msg) old_size = FileUtilities.getsizedir('/var/cache/dnf') args = ['--enablerepo=*', 'clean', 'all'] invalid = ['You need to be root', 'Cannot remove rpmdb file'] run_cleaner_cmd('dnf', args, '^unused regex$', invalid) new_size = FileUtilities.getsizedir('/var/cache/dnf') return old_size - new_size units = {"B": 1, "k": 10**3, "M": 10**6, "G": 10**9} def parseSize(size): """Parse the size returned by dnf""" number, unit = [string.strip() for string in size.split()] return int(float(number)*units[unit]) def dnf_autoremove(): """Run 'dnf autoremove' and return size in bytes recovered.""" if os.path.exists('/var/run/dnf.pid'): msg = _( "%s cannot be cleaned because it is currently running. Close it, and try again.") % "Dnf" raise RuntimeError(msg) cmd = ['dnf', '-y', 'autoremove'] (rc, stdout, stderr) = General.run_external(cmd) freed_bytes = 0 allout = stdout + stderr if 'Error: This command has to be run under the root user.' in allout: raise RuntimeError('dnf autoremove >> requires root permissions') if rc > 0: raise RuntimeError('dnf raised error %s: %s' % (rc, stderr)) cregex = re.compile("Freed space: ([\d.]+[\s]+[BkMG])") match = cregex.search(allout) if match: freed_bytes = parseSize(match.group(1)) logger.debug( 'dnf_autoremove >> total freed bytes: %s', freed_bytes) return freed_bytes locales = Locales() bleachbit-4.4.2/bleachbit/FileUtilities.py0000775000175000017500000010506214144024253017432 0ustar fabiofabio# vim: ts=4:sw=4:expandtab # -*- coding: UTF-8 -*- # BleachBit # Copyright (C) 2008-2021 Andrew Ziem # https://www.bleachbit.org # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . """ File-related utilities """ import bleachbit from bleachbit import _ import atexit import errno import glob import locale import logging import os import os.path import random import re import stat import string import sys import subprocess import tempfile import time logger = logging.getLogger(__name__) if 'nt' == os.name: from pywintypes import error as pywinerror import win32file import bleachbit.Windows os_path_islink = os.path.islink os.path.islink = lambda path: os_path_islink( path) or bleachbit.Windows.is_junction(path) if 'posix' == os.name: from bleachbit.General import WindowsError pywinerror = WindowsError try: from scandir import walk if 'nt' == os.name: import scandir import bleachbit.Windows class _Win32DirEntryPython(scandir.Win32DirEntryPython): def is_symlink(self): return super(_Win32DirEntryPython, self).is_symlink() or bleachbit.Windows.is_junction(self.path) scandir.scandir = scandir.scandir_python scandir.DirEntry = scandir.Win32DirEntryPython = _Win32DirEntryPython except ImportError: if sys.version_info < (3, 5, 0): # Python 3.5 incorporated scandir logger.warning( 'scandir is not available, so falling back to slower os.walk()') from os import walk def open_files_linux(): return glob.iglob("/proc/*/fd/*") def get_filesystem_type(path): """ * Get file system type from the given path * return value: The tuple of (file_system_type, device_name) * @ file_system_type: vfat, ntfs, etc * @ device_name: C://, D://, etc """ try: import psutil except ImportError: logger.warning('To get the file system type from the given path, you need to install psutil package') return ("unknown", "none") partitions = { partition.mountpoint: (partition.fstype, partition.device) for partition in psutil.disk_partitions() } if path in partitions: return partitions[path] splitpath = path.split(os.sep) for i in range(0, len(splitpath)-1): path = os.sep.join(splitpath[:i]) + os.sep if path in partitions: return partitions[path] path = os.sep.join(splitpath[:i]) if path in partitions: return partitions[path] return ("unknown", "none") def open_files_lsof(run_lsof=None): if run_lsof is None: def run_lsof(): return subprocess.check_output(["lsof", "-Fn", "-n"]) for f in run_lsof().split("\n"): if f.startswith("n/"): yield f[1:] # Drop lsof's "n" def open_files(): if sys.platform.startswith('linux'): files = open_files_linux() elif 'darwin' == sys.platform or sys.platform.startswith('freebsd'): files = open_files_lsof() else: raise RuntimeError('unsupported platform for open_files()') for filename in files: try: target = os.path.realpath(filename) except TypeError: # happens, for example, when link points to # '/etc/password\x00 (deleted)' continue else: yield target class OpenFiles: """Cached way to determine whether a file is open by active process""" def __init__(self): self.last_scan_time = None self.files = [] def file_qualifies(self, filename): """Return boolean whether filename qualifies to enter cache (check \ against blacklist)""" return not filename.startswith("/dev") and \ not filename.startswith("/proc") def scan(self): """Update cache""" self.last_scan_time = time.time() self.files = [] for filename in open_files(): if self.file_qualifies(filename): self.files.append(filename) def is_open(self, filename): """Return boolean whether filename is open by running process""" if self.last_scan_time is None or (time.time() - self.last_scan_time) > 10: self.scan() return os.path.realpath(filename) in self.files def __random_string(length): """Return random alphanumeric characters of given length""" return ''.join(random.choice(string.ascii_letters + '0123456789_.-') for i in range(length)) def bytes_to_human(bytes_i): # type: (int) -> str """Display a file size in human terms (megabytes, etc.) using preferred standard (SI or IEC)""" if bytes_i < 0: return '-' + bytes_to_human(-bytes_i) from bleachbit.Options import options if options.get('units_iec'): prefixes = ['', 'Ki', 'Mi', 'Gi', 'Ti', 'Pi'] base = 1024.0 else: prefixes = ['', 'k', 'M', 'G', 'T', 'P'] base = 1000.0 assert(isinstance(bytes_i, int)) if 0 == bytes_i: return '0B' if bytes_i >= base ** 3: decimals = 2 elif bytes_i >= base: decimals = 1 else: decimals = 0 for exponent in range(0, len(prefixes)): if bytes_i < base: abbrev = round(bytes_i, decimals) suf = prefixes[exponent] return locale.str(abbrev) + suf + 'B' else: bytes_i /= base return 'A lot.' def children_in_directory(top, list_directories=False): """Iterate files and, optionally, subdirectories in directory""" if type(top) is tuple: for top_ in top: yield from children_in_directory(top_, list_directories) return for (dirpath, dirnames, filenames) in walk(top, topdown=False): if list_directories: for dirname in dirnames: yield os.path.join(dirpath, dirname) for filename in filenames: yield os.path.join(dirpath, filename) def clean_ini(path, section, parameter): """Delete sections and parameters (aka option) in the file""" def write(parser, ini_file): """ Reimplementation of the original RowConfigParser write function. This function is 99% same as its origin. The only change is removing a cast to str. This is needed to handle unicode chars. """ if parser._defaults: ini_file.write("[%s]\n" % "DEFAULT") for (key, value) in parser._defaults.items(): ini_file.write("%s = %s\n" % (key, str(value).replace('\n', '\n\t'))) ini_file.write("\n") for section in parser._sections: ini_file.write("[%s]\n" % section) for (key, value) in parser._sections[section].items(): if key == "__name__": continue if (value is not None) or (parser._optcre == parser.OPTCRE): # The line below is the only changed line of the original function. # This is the original line for reference: # key = " = ".join((key, str(value).replace('\n', '\n\t'))) key = " = ".join((key, value.replace('\n', '\n\t'))) ini_file.write("%s\n" % (key)) ini_file.write("\n") encoding = detect_encoding(path) or 'utf_8_sig' # read file to parser config = bleachbit.RawConfigParser() config.optionxform = lambda option: option config.write = write with open(path, 'r', encoding=encoding) as fp: config.read_file(fp) # change file changed = False if config.has_section(section): if parameter is None: changed = True config.remove_section(section) elif config.has_option(section, parameter): changed = True config.remove_option(section, parameter) # write file if changed: from bleachbit.Options import options fp.close() if options.get('shred'): delete(path, True) with open(path, 'w', encoding=encoding, newline='') as fp: config.write(config, fp) def clean_json(path, target): """Delete key in the JSON file""" import json changed = False targets = target.split('/') # read file to parser with open(path, 'r', encoding='utf-8') as f: js = json.load(f) # change file pos = js while True: new_target = targets.pop(0) if not isinstance(pos, dict): break if new_target in pos and len(targets) > 0: # descend pos = pos[new_target] elif new_target in pos: # delete terminal target changed = True del(pos[new_target]) else: # target not found break if 0 == len(targets): # target not found break if changed: from bleachbit.Options import options if options.get('shred'): delete(path, True) # write file with open(path, 'w', encoding='utf-8') as f: json.dump(js, f) def delete(path, shred=False, ignore_missing=False, allow_shred=True): """Delete path that is either file, directory, link or FIFO. If shred is enabled as a function parameter or the BleachBit global parameter, the path will be shredded unless allow_shred = False. """ from bleachbit.Options import options is_special = False path = extended_path(path) do_shred = allow_shred and (shred or options.get('shred')) if not os.path.lexists(path): if ignore_missing: return raise OSError(2, 'No such file or directory', path) if 'posix' == os.name: # With certain (relatively rare) files on Windows os.lstat() # may return Access Denied mode = os.lstat(path)[stat.ST_MODE] is_special = stat.S_ISFIFO(mode) or stat.S_ISLNK(mode) if is_special: os.remove(path) elif os.path.isdir(path): delpath = path if do_shred: if not is_dir_empty(path): # Avoid renaming non-empty directory like https://github.com/bleachbit/bleachbit/issues/783 logger.info(_("Directory is not empty: %s"), path) return delpath = wipe_name(path) try: os.rmdir(delpath) except OSError as e: # [Errno 39] Directory not empty # https://bugs.launchpad.net/bleachbit/+bug/1012930 if errno.ENOTEMPTY == e.errno: logger.info(_("Directory is not empty: %s"), path) elif errno.EBUSY == e.errno: if os.name == 'posix' and os.path.ismount(path): logger.info(_("Skipping mount point: %s"), path) else: logger.info(_("Device or resource is busy: %s"), path) else: raise except WindowsError as e: # WindowsError: [Error 145] The directory is not empty: # 'C:\\Documents and Settings\\username\\Local Settings\\Temp\\NAILogs' # Error 145 may happen if the files are scheduled for deletion # during reboot. if 145 == e.winerror: logger.info(_("Directory is not empty: %s"), path) else: raise elif os.path.isfile(path): # wipe contents if do_shred: try: wipe_contents(path) except pywinerror as e: # 2 = The system cannot find the file specified. # This can happen with a broken symlink # https://github.com/bleachbit/bleachbit/issues/195 if 2 != e.winerror: raise # If a broken symlink, try os.remove() below. except IOError as e: # permission denied (13) happens shredding MSIE 8 on Windows 7 logger.debug("IOError #%s shredding '%s'", e.errno, path, exc_info=True) # wipe name os.remove(wipe_name(path)) else: # unlink os.remove(path) elif os.path.islink(path): os.remove(path) else: logger.info(_("Special file type cannot be deleted: %s"), path) def detect_encoding(fn): """Detect the encoding of the file""" try: import chardet except ImportError: logger.warning( 'chardet module is not available to detect character encoding') return None with open(fn, 'rb') as f: if not hasattr(chardet, 'universaldetector'): # This method works on Ubuntu 16.04 with an older version of the module. rawdata = f.read() det = chardet.detect(rawdata) if det['confidence'] > 0.5: return det['encoding'] return None # This method is faster, but it requires a newer version of the module. detector = chardet.universaldetector.UniversalDetector() for line in f.readlines(): detector.feed(line) if detector.done: break detector.close() return detector.result['encoding'] def ego_owner(filename): """Return whether current user owns the file""" return os.lstat(filename).st_uid == os.getuid() def exists_in_path(filename): """Returns boolean whether the filename exists in the path""" delimiter = ':' if 'nt' == os.name: delimiter = ';' for dirname in os.getenv('PATH').split(delimiter): if os.path.exists(os.path.join(dirname, filename)): return True return False def exe_exists(pathname): """Returns boolean whether executable exists""" if os.path.isabs(pathname): return os.path.exists(pathname) else: return exists_in_path(pathname) def execute_sqlite3(path, cmds): """Execute 'cmds' on SQLite database 'path'""" import sqlite3 import contextlib with contextlib.closing(sqlite3.connect(path)) as conn: cursor = conn.cursor() # overwrites deleted content with zeros # https://www.sqlite.org/pragma.html#pragma_secure_delete from bleachbit.Options import options if options.get('shred'): cursor.execute('PRAGMA secure_delete=ON') for cmd in cmds.split(';'): try: cursor.execute(cmd) except sqlite3.OperationalError as exc: if str(exc).find('no such function: ') >= 0: # fixme: determine why randomblob and zeroblob are not # available logger.exception(exc.message) else: raise sqlite3.OperationalError( '%s: %s' % (exc, path)) except sqlite3.DatabaseError as exc: raise sqlite3.DatabaseError( '%s: %s' % (exc, path)) cursor.close() conn.commit() def expand_glob_join(pathname1, pathname2): """Join pathname1 and pathname1, expand pathname, glob, and return as list""" pathname3 = os.path.expanduser(os.path.expandvars( os.path.join(pathname1, pathname2))) ret = [pathname4 for pathname4 in glob.iglob(pathname3)] return ret def extended_path(path): """If applicable, return the extended Windows pathname""" if 'nt' == os.name: if path.startswith(r'\\?'): return path if path.startswith(r'\\'): return '\\\\?\\unc\\' + path[2:] return '\\\\?\\' + path return path def extended_path_undo(path): """""" if 'nt' == os.name: if path.startswith(r'\\?\unc'): return '\\' + path[7:] if path.startswith(r'\\?'): return path[4:] return path def free_space(pathname): """Return free space in bytes""" if 'nt' == os.name: import psutil return psutil.disk_usage(pathname).free mystat = os.statvfs(pathname) return mystat.f_bfree * mystat.f_bsize def getsize(path): """Return the actual file size considering spare files and symlinks""" if 'posix' == os.name: try: __stat = os.lstat(path) except OSError as e: # OSError: [Errno 13] Permission denied # can happen when a regular user is trying to find the size of /var/log/hp/tmp # where /var/log/hp is 0774 and /var/log/hp/tmp is 1774 if errno.EACCES == e.errno: return 0 raise return __stat.st_blocks * 512 if 'nt' == os.name: # On rare files os.path.getsize() returns access denied, so first # try FindFilesW. # Also, apply prefix to use extended-length paths to support longer # filenames. finddata = win32file.FindFilesW(extended_path(path)) if not finddata: # FindFilesW does not work for directories, so fall back to # getsize() return os.path.getsize(path) else: size = (finddata[0][4] * (0xffffffff + 1)) + finddata[0][5] return size return os.path.getsize(path) def getsizedir(path): """Return the size of the contents of a directory""" total_bytes = sum( getsize(node) for node in children_in_directory(path, list_directories=False) ) return total_bytes def globex(pathname, regex): """Yield a list of files with pathname and filter by regex""" if type(pathname) is tuple: for singleglob in pathname: yield from globex(singleglob, regex) else: for path in glob.iglob(pathname): if re.search(regex, path): yield path def guess_overwrite_paths(): """Guess which partitions to overwrite (to hide deleted files)""" # In case overwriting leaves large files, placing them in # ~/.config makes it easy to find them and clean them. ret = [] if 'posix' == os.name: home = os.path.expanduser('~/.cache') if not os.path.exists(home): home = os.path.expanduser("~") ret.append(home) if not same_partition(home, '/tmp/'): ret.append('/tmp') elif 'nt' == os.name: localtmp = os.path.expandvars('$TMP') if not os.path.exists(localtmp): logger.warning(_("The environment variable TMP refers to a directory that does not exist: %s"), localtmp) localtmp = None from bleachbit.Windows import get_fixed_drives for drive in get_fixed_drives(): if localtmp and same_partition(localtmp, drive): ret.append(localtmp) else: ret.append(drive) else: raise NotImplementedError('Unsupported OS in guess_overwrite_paths') return ret def human_to_bytes(human, hformat='si'): """Convert a string like 10.2GB into bytes. By default use SI standard (base 10). The format of the GNU command 'du' (base 2) also supported.""" if 'si' == hformat: base = 1000 suffixes = 'kMGTE' elif 'du' == hformat: base = 1024 suffixes = 'KMGTE' else: raise ValueError("Invalid format: '%s'" % hformat) matches = re.match(r'^(\d+(?:\.\d+)?) ?([' + suffixes + ']?)B?$', human) if matches is None: raise ValueError("Invalid input for '%s' (hformat='%s')" % (human, hformat)) (amount, suffix) = matches.groups() if '' == suffix: exponent = 0 else: exponent = suffixes.find(suffix) + 1 return int(float(amount) * base**exponent) def is_dir_empty(dirname): """Returns boolean whether directory is empty. It assumes the path exists and is a directory. """ if hasattr(os, 'scandir'): if sys.version_info < (3, 6, 0): # Python 3.5 added os.scandir() without context manager. for _ in os.scandir(dirname): return False else: # Python 3.6 added the context manager. with os.scandir(dirname) as it: for _entry in it: return False return True # This method is slower, but it works with Python 3.4. return len(os.listdir(dirname)) == 0 def listdir(directory): """Return full path of files in directory. Path may be a tuple of directories.""" if type(directory) is tuple: for dirname in directory: yield from listdir(dirname) return dirname = os.path.expanduser(directory) if not os.path.lexists(dirname): return for filename in os.listdir(dirname): yield os.path.join(dirname, filename) def same_partition(dir1, dir2): """Are both directories on the same partition?""" if 'nt' == os.name: try: return free_space(dir1) == free_space(dir2) except pywinerror as e: if 5 == e.winerror: # Microsoft Office 2010 Starter Edition has a virtual # drive that gives access denied # https://bugs.launchpad.net/bleachbit/+bug/1372179 # https://bugs.launchpad.net/bleachbit/+bug/1474848 # https://github.com/az0/bleachbit/issues/27 return dir1[0] == dir2[0] raise stat1 = os.statvfs(dir1) stat2 = os.statvfs(dir2) return stat1[stat.ST_DEV] == stat2[stat.ST_DEV] def sync(): """Flush file system buffers. sync() is different than fsync()""" if 'posix' == os.name: import ctypes rc = ctypes.cdll.LoadLibrary('libc.so.6').sync() if 0 != rc: logger.error('sync() returned code %d', rc) elif 'nt' == os.name: import ctypes ctypes.cdll.LoadLibrary('msvcrt.dll')._flushall() def truncate_f(f): """Truncate the file object""" try: f.truncate(0) f.flush() os.fsync(f.fileno()) except OSError as e: if e.errno != errno.ENOSPC: raise def uris_to_paths(file_uris): """Return a list of paths from text/uri-list""" import urllib.parse import urllib.request assert isinstance(file_uris, (tuple, list)) file_paths = [] for file_uri in file_uris: if not file_uri: # ignore blank continue parsed_uri = urllib.parse.urlparse(file_uri) if parsed_uri.scheme == 'file': file_path = urllib.request.url2pathname(parsed_uri.path) if file_path[2] == ':': # remove front slash for Windows-style path file_path = file_path[1:] file_paths.append(file_path) else: logger.warning('Unsupported scheme: %s', file_uri) return file_paths def whitelisted_posix(path, check_realpath=True): """Check whether this POSIX path is whitelisted""" from bleachbit.Options import options if check_realpath and os.path.islink(path): # also check the link name if whitelisted_posix(path, False): return True # resolve symlink path = os.path.realpath(path) for pathname in options.get_whitelist_paths(): if pathname[0] == 'file' and path == pathname[1]: return True if pathname[0] == 'folder': if path == pathname[1]: return True if path.startswith(pathname[1] + os.sep): return True return False def whitelisted_windows(path): """Check whether this Windows path is whitelisted""" from bleachbit.Options import options for pathname in options.get_whitelist_paths(): # Windows is case insensitive if pathname[0] == 'file' and path.lower() == pathname[1].lower(): return True if pathname[0] == 'folder': if path.lower() == pathname[1].lower(): return True if path.lower().startswith(pathname[1].lower() + os.sep): return True # Simple drive letter like C:\ matches everything below if len(pathname[1]) == 3 and path.lower().startswith(pathname[1].lower()): return True return False if 'nt' == os.name: whitelisted = whitelisted_windows else: whitelisted = whitelisted_posix def wipe_contents(path, truncate=True): """Wipe files contents http://en.wikipedia.org/wiki/Data_remanence 2006 NIST Special Publication 800-88 (p. 7): "Studies have shown that most of today's media can be effectively cleared by one overwrite" """ def wipe_write(): size = getsize(path) try: f = open(path, 'wb') except IOError as e: if e.errno == errno.EACCES: # permission denied os.chmod(path, 0o200) # user write only f = open(path, 'wb') else: raise blanks = b'\0' * 4096 while size > 0: f.write(blanks) size -= 4096 f.flush() # flush to OS buffer os.fsync(f.fileno()) # force write to disk return f if 'nt' == os.name: from win32com.shell.shell import IsUserAnAdmin if 'nt' == os.name and IsUserAnAdmin(): from bleachbit.WindowsWipe import file_wipe, UnsupportedFileSystemError import warnings from bleachbit import _ try: file_wipe(path) except pywinerror as e: # 32=The process cannot access the file because it is being used by another process. # 33=The process cannot access the file because another process has # locked a portion of the file. if not e.winerror in (32, 33): # handle only locking errors raise # Try to truncate the file. This makes the behavior consistent # with Linux and with Windows when IsUserAdmin=False. try: with open(path, 'w') as f: truncate_f(f) except IOError as e2: if errno.EACCES == e2.errno: # Common when the file is locked # Errno 13 Permission Denied pass # translate exception to mark file to deletion in Command.py raise WindowsError(e.winerror, e.strerror) except UnsupportedFileSystemError as e: warnings.warn( _('There was at least one file on a file system that does not support advanced overwriting.'), UserWarning) f = wipe_write() else: # The wipe succeed, so prepare to truncate. f = open(path, 'w') else: f = wipe_write() if truncate: truncate_f(f) f.close() def wipe_name(pathname1): """Wipe the original filename and return the new pathname""" (head, _tail) = os.path.split(pathname1) # reference http://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits maxlen = 226 # first, rename to a long name i = 0 while True: try: pathname2 = os.path.join(head, __random_string(maxlen)) os.rename(pathname1, pathname2) break except OSError: if maxlen > 10: maxlen -= 10 i += 1 if i > 100: logger.info('exhausted long rename: %s', pathname1) pathname2 = pathname1 break # finally, rename to a short name i = 0 while True: try: pathname3 = os.path.join(head, __random_string(i + 1)) os.rename(pathname2, pathname3) break except: i += 1 if i > 100: logger.info('exhausted short rename: %s', pathname2) pathname3 = pathname2 break return pathname3 def wipe_path(pathname, idle=False): """Wipe the free space in the path This function uses an iterator to update the GUI.""" def temporaryfile(): # reference # http://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits maxlen = 185 f = None while True: try: f = tempfile.NamedTemporaryFile( dir=pathname, suffix=__random_string(maxlen), delete=False) # In case the application closes prematurely, make sure this # file is deleted atexit.register( delete, f.name, allow_shred=False, ignore_missing=True) break except OSError as e: if e.errno in (errno.ENAMETOOLONG, errno.ENOSPC, errno.ENOENT, errno.EINVAL): # ext3 on Linux 3.5 returns ENOSPC if the full path is greater than 264. # Shrinking the size helps. # Microsoft Windows returns ENOENT "No such file or directory" # or EINVAL "Invalid argument" # when the path is too long such as %TEMP% but not in C:\ if maxlen > 5: maxlen -= 5 continue raise return f def estimate_completion(): """Return (percent, seconds) to complete""" remaining_bytes = free_space(pathname) done_bytes = start_free_bytes - remaining_bytes if done_bytes < 0: # maybe user deleted large file after starting wipe done_bytes = 0 if 0 == start_free_bytes: done_percent = 0 else: done_percent = 1.0 * done_bytes / (start_free_bytes + 1) done_time = time.time() - start_time rate = done_bytes / (done_time + 0.0001) # bytes per second remaining_seconds = int(remaining_bytes / (rate + 0.0001)) return 1, done_percent, remaining_seconds logger.debug(_("Wiping path: %s") % pathname) files = [] total_bytes = 0 start_free_bytes = free_space(pathname) start_time = time.time() # Because FAT32 has a maximum file size of 4,294,967,295 bytes, # this loop is sometimes necessary to create multiple files. while True: try: logger.debug( _('Creating new, temporary file for wiping free space.')) f = temporaryfile() except OSError as e: # Linux gives errno 24 # Windows gives errno 28 No space left on device if e.errno in (errno.EMFILE, errno.ENOSPC): break else: raise # Get the file system type from the given path fstype = get_filesystem_type(pathname) fstype = fstype[0] logging.debug('File System:' + fstype) # print(f.name) # Added by Marvin for debugging #issue 1051 last_idle = time.time() # Write large blocks to quickly fill the disk. blanks = b'\0' * 65536 writtensize = 0 while True: try: if fstype != 'vfat': f.write(blanks) # In the ubuntu system, the size of file should be less then 4GB. If not, there should be EFBIG error. # So the maximum file size should be less than or equal to "4GB - 65536byte". elif writtensize < 4 * 1024 * 1024 * 1024 - 65536: writtensize += f.write(blanks) else: break except IOError as e: if e.errno == errno.ENOSPC: if len(blanks) > 1: # Try writing smaller blocks blanks = blanks[0:len(blanks) // 2] else: break elif e.errno == errno.EFBIG: break else: raise if idle and (time.time() - last_idle) > 2: # Keep the GUI responding, and allow the user to abort. # Also display the ETA. yield estimate_completion() last_idle = time.time() # Write to OS buffer try: f.flush() except IOError as e: # IOError: [Errno 28] No space left on device # seen on Microsoft Windows XP SP3 with ~30GB free space but # not on another XP SP3 with 64MB free space if not e.errno == errno.ENOSPC: logger.error( _("Error #%d when flushing the file buffer." % e.errno)) os.fsync(f.fileno()) # write to disk # Remember to delete files.append(f) # For statistics total_bytes += f.tell() # If no bytes were written, then quit. # See https://github.com/bleachbit/bleachbit/issues/502 if start_free_bytes - total_bytes < 2: # Modified by Marvin to fix the issue #1051 [12/06/2020] break # sync to disk sync() # statistics elapsed_sec = time.time() - start_time rate_mbs = (total_bytes / (1000 * 1000)) / elapsed_sec logger.info(_('Wrote {files:,} files and {bytes:,} bytes in {seconds:,} seconds at {rate:.2f} MB/s').format( files=len(files), bytes=total_bytes, seconds=int(elapsed_sec), rate=rate_mbs)) # how much free space is left (should be near zero) if 'posix' == os.name: stats = os.statvfs(pathname) logger.info(_("{bytes:,} bytes and {inodes:,} inodes available to non-super-user").format( bytes=stats.f_bsize * stats.f_bavail, inodes=stats.f_favail)) logger.info(_("{bytes:,} bytes and {inodes:,} inodes available to super-user").format( bytes=stats.f_bsize * stats.f_bfree, inodes=stats.f_ffree)) # truncate and close files for f in files: truncate_f(f) while True: try: # Nikita: I noticed a bug that prevented file handles from # being closed on FAT32. It sometimes takes two .close() calls # to do actually close (and therefore delete) a temporary file f.close() break except IOError as e: if e.errno == 0: logger.debug( _("Handled unknown error #0 while truncating file.")) time.sleep(0.1) # explicitly delete delete(f.name, ignore_missing=True) def vacuum_sqlite3(path): """Vacuum SQLite database""" execute_sqlite3(path, 'vacuum') openfiles = OpenFiles() bleachbit-4.4.2/bleachbit/markovify/0000775000175000017500000000000014144024254016306 5ustar fabiofabiobleachbit-4.4.2/bleachbit/markovify/chain.py0000664000175000017500000001204714144024253017745 0ustar fabiofabio# markovify # Copyright (c) 2015, Jeremy Singer-Vine # Origin: https://github.com/jsvine/markovify # MIT License: https://github.com/jsvine/markovify/blob/master/LICENSE.txt import random import operator import bisect import json # Python3 compatibility try: # pragma: no cover basestring except NameError: # pragma: no cover basestring = str BEGIN = "___BEGIN__" END = "___END__" def accumulate(iterable, func=operator.add): """ Cumulative calculations. (Summation, by default.) Via: https://docs.python.org/3/library/itertools.html#itertools.accumulate """ it = iter(iterable) total = next(it) yield total for element in it: total = func(total, element) yield total class Chain(object): """ A Markov chain representing processes that have both beginnings and ends. For example: Sentences. """ def __init__(self, corpus, state_size, model=None): """ `corpus`: A list of lists, where each outer list is a "run" of the process (e.g., a single sentence), and each inner list contains the steps (e.g., words) in the run. If you want to simulate an infinite process, you can come very close by passing just one, very long run. `state_size`: An integer indicating the number of items the model uses to represent its state. For text generation, 2 or 3 are typical. """ self.state_size = state_size self.model = model or self.build(corpus, self.state_size) self.precompute_begin_state() def build(self, corpus, state_size): """ Build a Python representation of the Markov model. Returns a dict of dicts where the keys of the outer dict represent all possible states, and point to the inner dicts. The inner dicts represent all possibilities for the "next" item in the chain, along with the count of times it appears. """ # Using a DefaultDict here would be a lot more convenient, however the memory # usage is far higher. model = {} for run in corpus: items = ([ BEGIN ] * state_size) + run + [ END ] for i in range(len(run) + 1): state = tuple(items[i:i+state_size]) follow = items[i+state_size] if state not in model: model[state] = {} if follow not in model[state]: model[state][follow] = 0 model[state][follow] += 1 return model def precompute_begin_state(self): """ Caches the summation calculation and available choices for BEGIN * state_size. Significantly speeds up chain generation on large corpuses. Thanks, @schollz! """ begin_state = tuple([ BEGIN ] * self.state_size) choices, weights = zip(*self.model[begin_state].items()) cumdist = list(accumulate(weights)) self.begin_cumdist = cumdist self.begin_choices = choices def move(self, state): """ Given a state, choose the next item at random. """ if state == tuple([ BEGIN ] * self.state_size): choices = self.begin_choices cumdist = self.begin_cumdist else: choices, weights = zip(*self.model[state].items()) cumdist = list(accumulate(weights)) r = random.random() * cumdist[-1] selection = choices[bisect.bisect(cumdist, r)] return selection def gen(self, init_state=None): """ Starting either with a naive BEGIN state, or the provided `init_state` (as a tuple), return a generator that will yield successive items until the chain reaches the END state. """ state = init_state or (BEGIN,) * self.state_size while True: next_word = self.move(state) if next_word == END: break yield next_word state = tuple(state[1:]) + (next_word,) def walk(self, init_state=None): """ Return a list representing a single run of the Markov model, either starting with a naive BEGIN state, or the provided `init_state` (as a tuple). """ return list(self.gen(init_state)) def to_json(self): """ Dump the model as a JSON object, for loading later. """ return json.dumps(list(self.model.items())) @classmethod def from_json(cls, json_thing): """ Given a JSON object or JSON string that was created by `self.to_json`, return the corresponding markovify.Chain. """ if isinstance(json_thing, basestring): obj = json.loads(json_thing) else: obj = json_thing if isinstance(obj, list): rehydrated = {tuple(item[0]): item[1] for item in obj} elif isinstance(obj, dict): rehydrated = obj else: raise ValueError("Object should be dict or list") state_size = len(list(rehydrated.keys())[0]) inst = cls(None, state_size, rehydrated) return inst bleachbit-4.4.2/bleachbit/markovify/splitters.py0000664000175000017500000000425614144024253020717 0ustar fabiofabio# -*- coding: utf-8 -*- # markovify # Copyright (c) 2015, Jeremy Singer-Vine # Origin: https://github.com/jsvine/markovify # MIT License: https://github.com/jsvine/markovify/blob/master/LICENSE.txt import re ascii_lowercase = "abcdefghijklmnopqrstuvwxyz" ascii_uppercase = ascii_lowercase.upper() # States w/ with thanks to https://github.com/unitedstates/python-us # Titles w/ thanks to https://github.com/nytimes/emphasis and @donohoe abbr_capped = "|".join([ "ala|ariz|ark|calif|colo|conn|del|fla|ga|ill|ind|kan|ky|la|md|mass|mich|minn|miss|mo|mont|neb|nev|okla|ore|pa|tenn|vt|va|wash|wis|wyo", # States "u.s", "mr|ms|mrs|msr|dr|gov|pres|sen|sens|rep|reps|prof|gen|messrs|col|sr|jf|sgt|mgr|fr|rev|jr|snr|atty|supt", # Titles "ave|blvd|st|rd|hwy", # Streets "jan|feb|mar|apr|jun|jul|aug|sep|sept|oct|nov|dec", # Months "|".join(ascii_lowercase) # Initials ]).split("|") abbr_lowercase = "etc|v|vs|viz|al|pct" exceptions = "U.S.|U.N.|E.U.|F.B.I.|C.I.A.".split("|") def is_abbreviation(dotted_word): clipped = dotted_word[:-1] if clipped[0] in ascii_uppercase: if clipped.lower() in abbr_capped: return True else: return False else: if clipped in abbr_lowercase: return True else: return False def is_sentence_ender(word): if word in exceptions: return False if word[-1] in [ "?", "!" ]: return True if len(re.sub(r"[^A-Z]", "", word)) > 1: return True if word[-1] == "." and (not is_abbreviation(word)): return True return False def split_into_sentences(text): potential_end_pat = re.compile(r"".join([ r"([\w\.'’&\]\)]+[\.\?!])", # A word that ends with punctuation r"([‘’“”'\"\)\]]*)", # Followed by optional quote/parens/etc r"(\s+(?![a-z\-–—]))", # Followed by whitespace + non-(lowercase or dash) ]), re.U) dot_iter = re.finditer(potential_end_pat, text) end_indices = [ (x.start() + len(x.group(1)) + len(x.group(2))) for x in dot_iter if is_sentence_ender(x.group(1)) ] spans = zip([None] + end_indices, end_indices + [None]) sentences = [ text[start:end].strip() for start, end in spans ] return sentences bleachbit-4.4.2/bleachbit/markovify/text.py0000664000175000017500000002101414144024253017641 0ustar fabiofabio# markovify # Copyright (c) 2015, Jeremy Singer-Vine # Origin: https://github.com/jsvine/markovify # MIT License: https://github.com/jsvine/markovify/blob/master/LICENSE.txt import re import random from .splitters import split_into_sentences from .chain import Chain, BEGIN # BleachBit does not use unidecode #from unidecode import unidecode DEFAULT_MAX_OVERLAP_RATIO = 0.7 DEFAULT_MAX_OVERLAP_TOTAL = 15 DEFAULT_TRIES = 10 class ParamError(Exception): pass class Text(object): def __init__(self, input_text, state_size=2, chain=None, parsed_sentences=None, retain_original=True): """ input_text: A string. state_size: An integer, indicating the number of words in the model's state. chain: A trained markovify.Chain instance for this text, if pre-processed. parsed_sentences: A list of lists, where each outer list is a "run" of the process (e.g. a single sentence), and each inner list contains the steps (e.g. words) in the run. If you want to simulate an infinite process, you can come very close by passing just one, very long run. """ can_make_sentences = parsed_sentences is not None or input_text is not None self.retain_original = retain_original and can_make_sentences self.state_size = state_size if self.retain_original: # not used in BleachBit pass else: if not chain: # not used in BleachBit pass self.chain = chain or Chain(parsed, state_size) def to_dict(self): """ Returns the underlying data as a Python dict. """ # not used in BleachBit pass def to_json(self): """ Returns the underlying data as a JSON string. """ # not used in BleachBit pass @classmethod def from_dict(cls, obj, **kwargs): return cls( None, state_size=obj["state_size"], chain=Chain.from_json(obj["chain"]), parsed_sentences=obj.get("parsed_sentences") ) @classmethod def from_json(cls, json_str): # not used in BleachBit pass def sentence_split(self, text): """ Splits full-text string into a list of sentences. """ return split_into_sentences(text) def sentence_join(self, sentences): """ Re-joins a list of sentences into the full text. """ return " ".join(sentences) word_split_pattern = re.compile(r"\s+") def word_split(self, sentence): """ Splits a sentence into a list of words. """ return re.split(self.word_split_pattern, sentence) def word_join(self, words): """ Re-joins a list of words into a sentence. """ return " ".join(words) def test_sentence_input(self, sentence): """ A basic sentence filter. This one rejects sentences that contain the type of punctuation that would look strange on its own in a randomly-generated sentence. """ # not used in BleachBit pass def generate_corpus(self, text): """ Given a text string, returns a list of lists; that is, a list of "sentences," each of which is a list of words. Before splitting into words, the sentences are filtered through `self.test_sentence_input` """ # not used in BleachBit pass def test_sentence_output(self, words, max_overlap_ratio, max_overlap_total): """ Given a generated list of words, accept or reject it. This one rejects sentences that too closely match the original text, namely those that contain any identical sequence of words of X length, where X is the smaller number of (a) `max_overlap_ratio` (default: 0.7) of the total number of words, and (b) `max_overlap_total` (default: 15). """ # not used in BleachBit pass def make_sentence(self, init_state=None, **kwargs): """ Attempts `tries` (default: 10) times to generate a valid sentence, based on the model and `test_sentence_output`. Passes `max_overlap_ratio` and `max_overlap_total` to `test_sentence_output`. If successful, returns the sentence as a string. If not, returns None. If `init_state` (a tuple of `self.chain.state_size` words) is not specified, this method chooses a sentence-start at random, in accordance with the model. If `test_output` is set as False then the `test_sentence_output` check will be skipped. If `max_words` is specified, the word count for the sentence will be evaluated against the provided limit. """ tries = kwargs.get('tries', DEFAULT_TRIES) mor = kwargs.get('max_overlap_ratio', DEFAULT_MAX_OVERLAP_RATIO) mot = kwargs.get('max_overlap_total', DEFAULT_MAX_OVERLAP_TOTAL) test_output = kwargs.get('test_output', True) max_words = kwargs.get('max_words', None) if init_state != None: prefix = list(init_state) for word in prefix: if word == BEGIN: prefix = prefix[1:] else: break else: prefix = [] for _ in range(tries): words = prefix + self.chain.walk(init_state) if max_words != None and len(words) > max_words: continue if test_output and hasattr(self, "rejoined_text"): if self.test_sentence_output(words, mor, mot): return self.word_join(words) else: return self.word_join(words) return None def make_short_sentence(self, max_chars, min_chars=0, **kwargs): """ Tries making a sentence of no more than `max_chars` characters and optionally no less than `min_chars` charcaters, passing **kwargs to `self.make_sentence`. """ tries = kwargs.get('tries', DEFAULT_TRIES) for _ in range(tries): sentence = self.make_sentence(**kwargs) if sentence and len(sentence) <= max_chars and len(sentence) >= min_chars: return sentence def make_sentence_with_start(self, beginning, strict=True, **kwargs): """ Tries making a sentence that begins with `beginning` string, which should be a string of one to `self.state` words known to exist in the corpus. If strict == True, then markovify will draw its initial inspiration only from sentences that start with the specified word/phrase. If strict == False, then markovify will draw its initial inspiration from any sentence containing the specified word/phrase. **kwargs are passed to `self.make_sentence` """ split = tuple(self.word_split(beginning)) word_count = len(split) if word_count == self.state_size: init_states = [ split ] elif word_count > 0 and word_count < self.state_size: if strict: init_states = [ (BEGIN,) * (self.state_size - word_count) + split ] else: init_states = [ key for key in self.chain.model.keys() # check for starting with begin as well ordered lists if tuple(filter(lambda x: x != BEGIN, key))[:word_count] == split ] random.shuffle(init_states) else: err_msg = "`make_sentence_with_start` for this model requires a string containing 1 to {0} words. Yours has {1}: {2}".format(self.state_size, word_count, str(split)) raise ParamError(err_msg) for init_state in init_states: output = self.make_sentence(init_state, **kwargs) if output is not None: return output return None @classmethod def from_chain(cls, chain_json, corpus=None, parsed_sentences=None): """ Init a Text class based on an existing chain JSON string or object If corpus is None, overlap checking won't work. """ chain = Chain.from_json(chain_json) return cls(corpus or None, parsed_sentences=parsed_sentences, state_size=chain.state_size, chain=chain) class NewlineText(Text): """ A (usable) example of subclassing markovify.Text. This one lets you markovify text where the sentences are separated by newlines instead of ". " """ def sentence_split(self, text): return re.split(r"\s*\n\s*", text) bleachbit-4.4.2/bleachbit/markovify/utils.py0000664000175000017500000000403114144024253020015 0ustar fabiofabio# markovify # Copyright (c) 2015, Jeremy Singer-Vine # Origin: https://github.com/jsvine/markovify # MIT License: https://github.com/jsvine/markovify/blob/master/LICENSE.txt from .chain import Chain from .text import Text def get_model_dict(thing): if isinstance(thing, Chain): return thing.model if isinstance(thing, Text): return thing.chain.model if isinstance(thing, list): return dict(thing) if isinstance(thing, dict): return thing raise ValueError("`models` should be instances of list, dict, markovify.Chain, or markovify.Text") def combine(models, weights=None): if weights is None: weights = [ 1 for _ in range(len(models)) ] if len(models) != len(weights): raise ValueError("`models` and `weights` lengths must be equal.") model_dicts = list(map(get_model_dict, models)) state_sizes = [ len(list(md.keys())[0]) for md in model_dicts ] if len(set(state_sizes)) != 1: raise ValueError("All `models` must have the same state size.") if len(set(map(type, models))) != 1: raise ValueError("All `models` must be of the same type.") c = {} for m, w in zip(model_dicts, weights): for state, options in m.items(): current = c.get(state, {}) for subseq_k, subseq_v in options.items(): subseq_prev = current.get(subseq_k, 0) current[subseq_k] = subseq_prev + (subseq_v * w) c[state] = current ret_inst = models[0] if isinstance(ret_inst, Chain): return Chain.from_json(c) if isinstance(ret_inst, Text): if not any(m.retain_original for m in models): return ret_inst.from_chain(c) combined_sentences = [] for m in models: if m.retain_original: combined_sentences += m.parsed_sentences return ret_inst.from_chain(c, parsed_sentences=combined_sentences) if isinstance(ret_inst, list): return list(c.items()) if isinstance(ret_inst, dict): return c bleachbit-4.4.2/bleachbit/markovify/__init__.py0000664000175000017500000000030214144024253020411 0ustar fabiofabio# version is not needed #from .__version__ import __version__ from .chain import Chain from .text import Text, NewlineText from .splitters import split_into_sentences from .utils import combine bleachbit-4.4.2/bleachbit/DeepScan.py0000775000175000017500000000767114144024253016350 0ustar fabiofabio# vim: ts=4:sw=4:expandtab # -*- coding: UTF-8 -*- # BleachBit # Copyright (C) 2008-2021 Andrew Ziem # https://www.bleachbit.org # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . """ Scan directory tree for files to delete """ import logging import os import platform import re import unicodedata from collections import namedtuple from bleachbit import fs_scan_re_flags from . import Command def normalized_walk(top, **kwargs): """ macOS uses decomposed UTF-8 to store filenames. This functions is like `os.walk` but recomposes those decomposed filenames on macOS """ try: from scandir import walk except: # there is a warning in FileUtilities, so don't warn again here from os import walk if 'Darwin' == platform.system(): for dirpath, dirnames, filenames in walk(top, **kwargs): yield dirpath, dirnames, [ unicodedata.normalize('NFC', fn) for fn in filenames ] else: yield from walk(top, **kwargs) Search = namedtuple('Search', ['command', 'regex', 'nregex', 'wholeregex', 'nwholeregex']) Search.__new__.__defaults__ = (None,) * len(Search._fields) class CompiledSearch: """Compiled search condition""" def __init__(self, search): self.command = search.command def re_compile(regex): return re.compile(regex, fs_scan_re_flags) if regex else None self.regex = re_compile(search.regex) self.nregex = re_compile(search.nregex) self.wholeregex = re_compile(search.wholeregex) self.nwholeregex = re_compile(search.nwholeregex) def match(self, dirpath, filename): full_path = os.path.join(dirpath, filename) if self.regex and not self.regex.search(filename): return None if self.nregex and self.nregex.search(filename): return None if self.wholeregex and not self.wholeregex.search(full_path): return None if self.nwholeregex and self.nwholeregex.search(full_path): return None return full_path class DeepScan: """Advanced directory tree scan""" def __init__(self, searches): self.roots = [] self.searches = searches def scan(self): """Perform requested searches and yield each match""" logging.getLogger(__name__).debug( 'DeepScan.scan: searches=%s', str(self.searches)) import time yield_time = time.time() for (top, searches) in self.searches.items(): compiled_searches = [CompiledSearch(s) for s in searches] for (dirpath, _dirnames, filenames) in normalized_walk(top): for c in compiled_searches: # fixme, don't match filename twice for filename in filenames: full_name = c.match(dirpath, filename) if full_name is not None: # fixme: support other commands if c.command == 'delete': yield Command.Delete(full_name) elif c.command == 'shred': yield Command.Shred(full_name) if time.time() - yield_time > 0.25: # allow GTK+ to process the idle loop yield True yield_time = time.time() bleachbit-4.4.2/bleachbit/GuiChaff.py0000775000175000017500000001634114144024253016334 0ustar fabiofabio# vim: ts=4:sw=4:expandtab # -*- coding: UTF-8 -*- # BleachBit # Copyright (C) 2008-2021 Andrew Ziem # https://www.bleachbit.org # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . """ GUI for making chaff """ from bleachbit import _ from bleachbit.Chaff import download_models, generate_emails, generate_2600, have_models from gi.repository import Gtk, GLib import logging import os logger = logging.getLogger(__name__) def make_files_thread(file_count, inspiration, output_folder, delete_when_finished, on_progress): if inspiration == 0: generated_file_names = generate_2600( file_count, output_folder, on_progress=on_progress) elif inspiration == 1: generated_file_names = generate_emails( file_count, output_folder, on_progress=on_progress) if delete_when_finished: on_progress(0, msg=_('Deleting files')) for i in range(0, file_count): os.unlink(generated_file_names[i]) on_progress(1.0 * (i+1)/file_count) on_progress(1.0, is_done=True) class ChaffDialog(Gtk.Dialog): """Present the dialog to make chaff""" def __init__(self, parent): self._make_dialog(parent) def _make_dialog(self, parent): """Make the main dialog""" # TRANSLATORS: BleachBit creates digital chaff like that is like the # physical chaff airplanes use to protect themselves from radar-guided # missiles. For more explanation, see the online documentation. Gtk.Dialog.__init__(self, _("Make chaff"), parent) self.set_border_width(5) box = self.get_content_area() label = Gtk.Label( _("Make randomly-generated messages inspired by documents.")) box.add(label) inspiration_box = Gtk.Box(orientation=Gtk.Orientation.HORIZONTAL) # TRANSLATORS: Inspiration is a choice of documents from which random # text will be generated. inspiration_box.add(Gtk.Label(_("Inspiration"))) self.inspiration_combo = Gtk.ComboBoxText() self.inspiration_combo_options = ( _('2600 Magazine'), _("Hillary Clinton's emails")) for combo_option in self.inspiration_combo_options: self.inspiration_combo.append_text(combo_option) self.inspiration_combo.set_active(0) # Set default inspiration_box.add(self.inspiration_combo) box.add(inspiration_box) spin_box = Gtk.Box(orientation=Gtk.Orientation.HORIZONTAL) spin_box.add(Gtk.Label(_("Number of files"))) adjustment = Gtk.Adjustment(100, 1, 99999, 1, 1000, 0) self.file_count = Gtk.SpinButton(adjustment=adjustment) spin_box.add(self.file_count) box.add(spin_box) folder_box = Gtk.Box(orientation=Gtk.Orientation.HORIZONTAL) folder_box.add(Gtk.Label(_("Select destination folder"))) self.choose_folder_button = Gtk.FileChooserButton() self.choose_folder_button.set_action( Gtk.FileChooserAction.SELECT_FOLDER) import tempfile self.choose_folder_button.set_filename(tempfile.gettempdir()) folder_box.add(self.choose_folder_button) box.add(folder_box) delete_box = Gtk.Box(orientation=Gtk.Orientation.HORIZONTAL) delete_box.add(Gtk.Label(_("When finished"))) self.when_finished_combo = Gtk.ComboBoxText() self.combo_options = ( _('Delete without shredding'), _('Do not delete')) for combo_option in self.combo_options: self.when_finished_combo.append_text(combo_option) self.when_finished_combo.set_active(0) # Set default delete_box.add(self.when_finished_combo) box.add(delete_box) self.progressbar = Gtk.ProgressBar() box.add(self.progressbar) self.progressbar.hide() self.make_button = Gtk.Button(_("Make files")) self.make_button.connect('clicked', self.on_make_files) box.add(self.make_button) def download_models_gui(self): """Download models and return whether successful as boolean""" def on_download_error(msg, msg2): dialog = Gtk.MessageDialog(self, 0, Gtk.MessageType.ERROR, Gtk.ButtonsType.CANCEL, msg) dialog.format_secondary_text(msg2) dialog.run() dialog.destroy() return download_models(on_error=on_download_error) def download_models_dialog(self): """Download models""" dialog = Gtk.MessageDialog(self, 0, Gtk.MessageType.QUESTION, Gtk.ButtonsType.OK_CANCEL, _("Download data needed for chaff generator?")) response = dialog.run() ret = None if response == Gtk.ResponseType.OK: # User wants to download ret = self.download_models_gui() # True if successful elif response == Gtk.ResponseType.CANCEL: ret = False dialog.destroy() return ret def on_make_files(self, widget): """Callback for make files button""" file_count = self.file_count.get_value_as_int() output_dir = self.choose_folder_button.get_filename() delete_when_finished = self.when_finished_combo.get_active() == 0 inspiration = self.inspiration_combo.get_active() if not output_dir: dialog = Gtk.MessageDialog(self, 0, Gtk.MessageType.ERROR, Gtk.ButtonsType.CANCEL, _("Select destination folder")) dialog.run() dialog.destroy() return if not have_models(): if not self.download_models_dialog(): return def _on_progress(fraction, msg, is_done): """Update progress bar from GLib main loop""" if msg: self.progressbar.set_text(msg) self.progressbar.set_fraction(fraction) if is_done: self.progressbar.hide() self.make_button.set_sensitive(True) def on_progress(fraction, msg=None, is_done=False): """Callback for progress bar""" # Use idle_add() because threads cannot make GDK calls. GLib.idle_add(_on_progress, fraction, msg, is_done) msg = _('Generating files') logger.info(msg) self.progressbar.show() self.progressbar.set_text(msg) self.progressbar.set_show_text(True) self.progressbar.set_fraction(0.0) self.make_button.set_sensitive(False) import threading args = (file_count, inspiration, output_dir, delete_when_finished, on_progress) t = threading.Thread(target=make_files_thread, args=args) t.start() def run(self): """Run the dialog""" self.show_all() bleachbit-4.4.2/bleachbit/WindowsWipe.py0000664000175000017500000012076714144024253017144 0ustar fabiofabio# vim: ts=4:sw=4:expandtab # BleachBit # Copyright (C) 2008-2021 Andrew Ziem # https://www.bleachbit.org # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from bleachbit.FileUtilities import extended_path, extended_path_undo """ *** *** Owner: Andrew Ziem *** Author: Peter Marshall *** *** References: *** Windows Internals (Russinovich, Solomon, Ionescu), 6th edition *** http://windowsitpro.com/systems-management/inside-windows-nt-disk-defragmenting *** https://technet.microsoft.com/en-us/sysinternals/sdelete.aspx *** https://blogs.msdn.microsoft.com/jeffrey_wall/2004/09/13/defrag-api-c-wrappers/ *** https://msdn.microsoft.com/en-us/library/windows/desktop/aa364572(v=vs.85).aspx *** *** *** Algorithm *** --Phase 1 *** - Check if the file has special characteristics (sparse, encrypted, *** compressed), determine file system (NTFS or FAT), Windows version. *** - Read the on-disk locations of the file using defrag API. *** - If file characteristics don't rule it out, just do a direct write *** of zero-fill on entire file size and flush to disk. *** - Read back the on-disk locations of the file using defrag API. *** - If locations are exactly the same, we are done. *** - Otherwise, enumerate clusters that did not get overwritten in place *** ("missed clusters"). *** They are probably still untouched, we need to wipe them. *** - If it was a special file that wouldn't be wiped by a direct write, *** we will truncate the file and treat it all as missed clusters. *** *** --Phase 2 *** - (*) Get volume bitmap of free/allocated clusters using defrag API. *** Figure out if checkpoint has made our missed clusters available *** for use again (this is potentially delayed by a few seconds in NTFS). *** - If they have not yet been made available, wait 0.1s then repeat *** previous check (*), up to a limit of 7s in polling. *** - Figure out if it is better to bridge the extents, wiping more clusters *** but gaining a performance boost from reduced total cycles and overhead. *** - Recurse over the extents we need to wipe, breaking them down into *** smaller extents if necessary. *** - Write a zero-fill file that will provide enough clusters to *** completely overwrite each extent in turn. *** - Iterate over the zero-fill file, moving clusters from our zero file *** to the missed clusters using defrag API. *** - If the defrag move operation did not succeed, it was probably because *** another process has grabbed a cluster on disk that we wanted to *** write to. This can also happen when, by chance, the move's source and *** target ranges overlap. *** - In response, we can break the extent down into sub-sections and *** attempt to wipe each subsection (eventually down to a granularity *** of one cluster). We also inspect allocated/free sectors to look ahead *** and avoid making move calls that we know will fail. *** - If a cluster was allocated by some other Windows process before we could *** explicitly wipe it, it is assumed to be wiped. Even if Windows writes a *** small amount of explicit data to a cluster, it seems to write zero-fill *** out to the end of the cluster to round it out. *** *** *** TO DO *** - Test working correctly if per-user disk quotas are in place *** """ # Imports. import sys import os import struct import logging from operator import itemgetter from random import randint from collections import namedtuple from win32api import (GetVolumeInformation, GetDiskFreeSpace, GetVersionEx, Sleep) from win32file import (CreateFile, CreateFileW, CloseHandle, GetDriveType, GetFileSize, GetFileAttributesW, SetFileAttributesW, DeviceIoControl, SetFilePointer, WriteFile, LockFile, DeleteFile, SetEndOfFile, FlushFileBuffers) from winioctlcon import (FSCTL_GET_RETRIEVAL_POINTERS, FSCTL_GET_VOLUME_BITMAP, FSCTL_GET_NTFS_VOLUME_DATA, FSCTL_MOVE_FILE, FSCTL_SET_COMPRESSION, FSCTL_SET_SPARSE, FSCTL_SET_ZERO_DATA) from win32file import (GENERIC_READ, GENERIC_WRITE, FILE_BEGIN, FILE_SHARE_READ, FILE_SHARE_WRITE, OPEN_EXISTING, CREATE_ALWAYS, FILE_FLAG_BACKUP_SEMANTICS, DRIVE_REMOTE, DRIVE_CDROM, DRIVE_UNKNOWN) from win32con import (FILE_ATTRIBUTE_ENCRYPTED, FILE_ATTRIBUTE_COMPRESSED, FILE_ATTRIBUTE_SPARSE_FILE, FILE_ATTRIBUTE_HIDDEN, FILE_ATTRIBUTE_READONLY, FILE_FLAG_RANDOM_ACCESS, FILE_FLAG_NO_BUFFERING, FILE_FLAG_WRITE_THROUGH, COMPRESSION_FORMAT_DEFAULT) VER_SUITE_PERSONAL = 0x200 # doesn't seem to be present in win32con. # Constants. simulate_concurrency = False # remove this test function when QA complete # drive_letter_safety = "E" # protection to only use removeable drives # don't use C: or D:, but E: and beyond OK. tmp_file_name = "bbtemp.dat" spike_file_name = "bbspike" # cluster number will be appended write_buf_size = 512 * 1024 # 512 kilobytes # Set up logging logger = logging.getLogger(__name__) # Unpacks the next element in a structure, using format requested. # Returns the element and the remaining content of the structure. def unpack_element(fmt, structure): chunk_size = struct.calcsize(fmt) element = struct.unpack(fmt, structure[:chunk_size]) if element and len(element) > 0: element = element[0] # convert from tuple to single element structure = structure[chunk_size:] return element, structure # GET_RETRIEVAL_POINTERS gives us a list of VCN, LCN tuples. # Convert from that format into a list of cluster start/end tuples. # The flag for writing bridged extents is a way of handling # the structure of compressed files. If a part of the file is close # to contiguous on disk, bridge its extents to combine them, even # though there are some unrelated clusters in between. # Generator function, will return results one tuple at a time. def logical_ranges_to_extents(ranges, bridge_compressed=False): if not bridge_compressed: vcn_count = 0 for vcn, lcn in ranges: # If we encounter an LCN of -1, we have reached a # "space-saved" part of a compressed file. These clusters # don't map to clusters on disk, just advance beyond them. if lcn < 0: vcn_count = vcn continue # Figure out length for this cluster range. # Keep track of VCN inside this file. this_vcn_span = vcn - vcn_count vcn_count = vcn assert this_vcn_span >= 0 yield (lcn, lcn + this_vcn_span - 1) else: vcn_count = 0 last_record = len(ranges) index = 0 while index < last_record: vcn, lcn = ranges[index] # If we encounter an LCN of -1, we have reached a # "space-saved" part of a compressed file. These clusters # don't map to clusters on disk, just advance beyond them. if lcn < 0: vcn_count = vcn index += 1 continue # Figure out if we have a block of clusters that can # be merged together. The pattern is regular disk # clusters interspersed with -1 space-saver sections # that are arranged with gaps of 16 clusters or less. merge_index = index while (lcn >= 0 and merge_index + 2 < last_record and ranges[merge_index + 1][1] < 0 and ranges[merge_index + 2][1] >= 0 and ranges[merge_index + 2][1] - ranges[merge_index][1] <= 16 and ranges[merge_index + 2][1] - ranges[merge_index][1] > 0): merge_index += 2 # Figure out length for this cluster range. # Keep track of VCN inside this file. if merge_index == index: index += 1 this_vcn_span = vcn - vcn_count vcn_count = vcn assert this_vcn_span >= 0 yield (lcn, lcn + this_vcn_span - 1) else: index = merge_index + 1 last_vcn_span = (ranges[merge_index][0] - ranges[merge_index - 1][0]) vcn = ranges[merge_index][0] vcn_count = vcn assert last_vcn_span >= 0 yield (lcn, ranges[merge_index][1] + last_vcn_span - 1) # Determine clusters that are in extents list A but not in B. # Generator function, will return results one tuple at a time. def extents_a_minus_b(a, b): # Sort the lists of start/end points. a_sorted = sorted(a, key=itemgetter(0)) b_sorted = sorted(b, key=itemgetter(0)) b_is_empty = not b for a_begin, a_end in a_sorted: # If B is an empty list, each item of A will be unchanged. if b_is_empty: yield (a_begin, a_end) for b_begin, b_end in b_sorted: if b_begin > a_end: # Already gone beyond current A range and no matches. # Return this range of A unbroken. yield (a_begin, a_end) break elif b_end < a_begin: # Too early in list, keep searching. continue elif b_begin <= a_begin: if b_end >= a_end: # This range of A is completely covered by B. # Do nothing and pass on to next range of A. break else: # This range of A is partially covered by B. # Remove the covered range from A and loop a_begin = b_end + 1 else: # This range of A is partially covered by B. # Return the first part of A not covered. # Either process remainder of A range or move to next A. yield (a_begin, b_begin - 1) if b_end >= a_end: break else: a_begin = b_end + 1 # Decide if it will be more efficient to bridge the extents and wipe # some additional clusters that weren't strictly part of the file. # By grouping write/move cycles into larger portions, we can reduce # overhead and complete the wipe quicker - even though it involves # a higher number of total clusters written. def choose_if_bridged(volume_handle, total_clusters, orig_extents, bridged_extents): logger.debug('bridged extents: {}'.format(bridged_extents)) allocated_extents = [] volume_bitmap, bitmap_size = get_volume_bitmap(volume_handle, total_clusters) count_ofree, count_oallocated = check_extents( orig_extents, volume_bitmap) count_bfree, count_ballocated = check_extents( bridged_extents, volume_bitmap, allocated_extents) bridged_extents = [x for x in extents_a_minus_b(bridged_extents, allocated_extents)] extra_allocated_clusters = count_ballocated - count_oallocated saving_in_extents = len(orig_extents) - len(bridged_extents) logger.debug(("Bridged extents would require us to work around %d " + "more allocated clusters.") % extra_allocated_clusters) logger.debug("It would reduce extent count from %d to %d." % ( len(orig_extents), len(bridged_extents))) # Use a penalty of 10 extents for each extra allocated cluster. # Why 10? Assuming our next granularity above 1 cluster is a 10 cluster # extent, a single allocated cluster would cause us to perform 8 # additional write/move cycles due to splitting that extent into single # clusters. # If we had a notion of distribution of extra allocated clusters, # we could make this calc more exact. But it's just a rule of thumb. tradeoff = saving_in_extents - extra_allocated_clusters * 10 if tradeoff > 0: logger.debug("Quickest method should be bridged extents") return bridged_extents else: logger.debug("Quickest method should be original extents") return orig_extents # Break an extent into smaller portions (numbers are tuned to give something # in the range of 8 to 15 portions). # Generator function, will return results one tuple at a time. def split_extent(lcn_start, lcn_end): split_factor = 10 exponent = 0 count = lcn_end - lcn_start + 1 while count > split_factor**(exponent + 1.3): exponent += 1 extent_size = split_factor**exponent for x in range(lcn_start, lcn_end + 1, extent_size): yield (x, min(x + extent_size - 1, lcn_end)) # Check extents to see if they are marked as free. def check_extents(extents, volume_bitmap, allocated_extents=None): count_free, count_allocated = (0, 0) for lcn_start, lcn_end in extents: for cluster in range(lcn_start, lcn_end + 1): if check_mapped_bit(volume_bitmap, cluster): count_allocated += 1 if allocated_extents is not None: allocated_extents.append((cluster, cluster)) # Modified by Marvin [12/05/2020] The extents should have (start, end) format else: count_free += 1 logger.debug("Extents checked: clusters free %d; allocated %d", count_free, count_allocated) return (count_free, count_allocated) # Check extents to see if they are marked as free. # Copy of the above that simulates concurrency for testing purposes. # Once every x clusters at random it will allocate a cluster on disk # to prove that the algorithm can handle it. def check_extents_concurrency(extents, volume_bitmap, tmp_file_path, volume_handle, total_clusters, allocated_extents=None): odds_to_allocate = 1200 # 1 in 1200 count_free, count_allocated = (0, 0) for lcn_start, lcn_end in extents: for cluster in range(lcn_start, lcn_end + 1): # Every once in a while, occupy a particular cluster on disk. if randint(1, odds_to_allocate) == odds_to_allocate: spike_cluster(volume_handle, cluster, tmp_file_path) if bool(randint(0, 1)): # Simulate allocated before the check, by refetching # the volume bitmap. logger.debug("Simulate known allocated") volume_bitmap, _ = get_volume_bitmap( volume_handle, total_clusters) else: # Simulate allocated after the check. logger.debug("Simulate unknown allocated") if check_mapped_bit(volume_bitmap, cluster): count_allocated += 1 if allocated_extents is not None: allocated_extents.append(cluster) else: count_free += 1 logger.debug("Extents checked: clusters free %d; allocated %d", count_free, count_allocated) return (count_free, count_allocated) # Allocate a cluster on disk by pinning it with a file. # This simulates another process having grabbed it while our # algorithm is working. # This is only used for testing, especially testing concurrency issues. def spike_cluster(volume_handle, cluster, tmp_file_path): spike_file_path = os.path.dirname(tmp_file_path) if spike_file_path[-1] != os.sep: spike_file_path += os.sep spike_file_path += spike_file_name + str(cluster) file_handle = CreateFile(spike_file_path, GENERIC_READ | GENERIC_WRITE, FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, None, CREATE_ALWAYS, 0, None) # 2000 bytes is enough to direct the file to its own cluster and not # land entirely in the MFT. write_zero_fill(file_handle, 2000) move_file(volume_handle, file_handle, 0, cluster, 1) CloseHandle(file_handle) logger.debug("Spiked cluster %d with %s" % (cluster, spike_file_path)) # Check if an LCN is allocated (True) or free (False). # The LCN determines at what index into the bytes/bits structure we # should look. def check_mapped_bit(volume_bitmap, lcn): assert isinstance(lcn, int) mapped_bit = volume_bitmap[lcn // 8] bit_location = lcn % 8 # zero-based if bit_location > 0: mapped_bit = mapped_bit >> bit_location mapped_bit = mapped_bit & 1 return mapped_bit > 0 # Check the operating system. Go no further unless we are on # Windows and it's Win NT or later. def check_os(): if os.name.lower() != "nt": raise RuntimeError("This function requires Windows NT or later") # Determine which version of Windows we are running. # Not currently used, except to control encryption test case # depending on whether it's Windows Home Edition or something higher end. def determine_win_version(): ver_info = GetVersionEx(1) is_home = bool(ver_info[7] & VER_SUITE_PERSONAL) if ver_info[:2] == (6, 0): return "Vista", is_home elif ver_info[0] >= 6: return "Later than Vista", is_home else: return "Something else", is_home # Open the file to get a Windows file handle, ensuring it exists. # CreateFileW gives us Unicode support. def open_file(file_name, mode=GENERIC_READ): file_handle = CreateFileW(file_name, mode, 0, None, OPEN_EXISTING, FILE_FLAG_BACKUP_SEMANTICS, None) return file_handle # Close file def close_file(file_handle): CloseHandle(file_handle) # Get some basic information about a file. def get_file_basic_info(file_name, file_handle): file_attributes = GetFileAttributesW(file_name) file_size = GetFileSize(file_handle) is_compressed = bool(file_attributes & FILE_ATTRIBUTE_COMPRESSED) is_encrypted = bool(file_attributes & FILE_ATTRIBUTE_ENCRYPTED) is_sparse = bool(file_attributes & FILE_ATTRIBUTE_SPARSE_FILE) is_special = is_compressed | is_encrypted | is_sparse if is_special: logger.debug('{}: {} {} {}'.format(file_name, 'compressed' if is_compressed else '', 'encrypted' if is_encrypted else '', 'sparse' if is_sparse else '')) return file_size, is_special # Truncate a file. Do this when we want to release its clusters. def truncate_file(file_handle): SetFilePointer(file_handle, 0, FILE_BEGIN) SetEndOfFile(file_handle) FlushFileBuffers(file_handle) # Given a Windows file path, determine the volume that contains it. # Append the separator \ to it (more useful for subsequent calls). def volume_from_file(file_name): # strip \\?\ split_path = os.path.splitdrive(extended_path_undo(file_name)) volume = split_path[0] if volume and volume[-1] != os.sep: volume += os.sep return volume class UnsupportedFileSystemError(Exception): """An exception for an unsupported file system""" # Given a volume, get the relevant volume information. # We are interested in: # First call: Drive Name; Max Path; File System. # Second call: Sectors per Cluster; Bytes per Sector; Total # of Clusters. # Third call: Drive Type. def get_volume_information(volume): # If it's a UNC path, raise an error. if not volume: raise UnsupportedFileSystemError( "Only files with a Local File System path can be wiped.") result1 = GetVolumeInformation(volume) result2 = GetDiskFreeSpace(volume) result3 = GetDriveType(volume) for drive_enum, error_reason in [ (DRIVE_REMOTE, "a network drive"), (DRIVE_CDROM, "a CD-ROM"), (DRIVE_UNKNOWN, "an unknown drive type")]: if result3 == drive_enum: raise UnsupportedFileSystemError( "This file is on %s and can't be wiped." % error_reason) # Only NTFS and FAT variations are supported. # UDF (file system for CD-RW etc) is not supported. if result1[4].upper() == "UDF": raise UnsupportedFileSystemError( "This file system (UDF) is not supported.") volume_info = namedtuple('VolumeInfo', [ 'drive_name', 'max_path', 'file_system', 'sectors_per_cluster', 'bytes_per_sector', 'total_clusters']) return volume_info(result1[0], result1[2], result1[4], result2[0], result2[1], result2[3]) # Get read/write access to a volume. def obtain_readwrite(volume): # Optional protection that we are running on removable media only. assert volume # if drive_letter_safety: # drive_containing_file = volume[0].upper() # assert drive_containing_file >= drive_letter_safety.upper() volume = '\\\\.\\' + volume if volume[-1] == os.sep: volume = volume.rstrip(os.sep) # We need the FILE_SHARE flags so that this open call can succeed # despite something on the volume being in use by another process. volume_handle = CreateFile(volume, GENERIC_READ | GENERIC_WRITE, FILE_SHARE_READ | FILE_SHARE_WRITE, None, OPEN_EXISTING, FILE_FLAG_RANDOM_ACCESS | FILE_FLAG_NO_BUFFERING | FILE_FLAG_WRITE_THROUGH, None) #logger.debug("Opened volume %s", volume) return volume_handle # Retrieve a list of pointers to the file location on disk. # If translate_to_extents is False, return the Windows VCN/LCN format. # If True, do an extra conversion to get a list of extents on disk. def get_extents(file_handle, translate_to_extents=True): # Assemble input structure and query Windows for retrieval pointers. # The input structure is the number 0 as a signed 64 bit integer. input_struct = struct.pack('q', 0) # 4K, 32K, 256K, 2M step ups in buffer size, until call succeeds. # Compressed/encrypted/sparse files tend to have more chopped up extents. buf_retry_sizes = [4 * 1024, 32 * 1024, 256 * 1024, 2 * 1024**2] for retrieval_pointers_buf_size in buf_retry_sizes: try: rp_struct = DeviceIoControl(file_handle, FSCTL_GET_RETRIEVAL_POINTERS, input_struct, retrieval_pointers_buf_size) except: err_info = sys.exc_info()[1] err_code = err_info.winerror if err_code == 38: # when file size is 0. # (38, 'DeviceIoControl', 'Reached the end of the file.') return [] elif err_code in [122, 234]: # when buffer not large enough. # (122, 'DeviceIoControl', # 'The data area passed to a system call is too small.') # (234, 'DeviceIoControl', 'More data is available.') pass else: raise else: # Call succeeded, break out from for loop. break # At this point we have a FSCTL_GET_RETRIEVAL_POINTERS (rp) structure. # Process content of the first part of structure. # Separate the retrieval pointers list up front, so we are not making # too many string copies of it. chunk_size = struct.calcsize('IIq') rp_list = rp_struct[chunk_size:] rp_struct = rp_struct[:chunk_size] record_count, rp_struct = unpack_element('I', rp_struct) # 4 bytes _, rp_struct = unpack_element('I', rp_struct) # 4 bytes starting_vcn, rp_struct = unpack_element('q', rp_struct) # 8 bytes # 4 empty bytes were consumed above. # This is for reasons of 64-bit alignment inside structure. # If we make the GET_RETRIEVAL_POINTERS request with 0, # this should always come back 0. assert starting_vcn == 0 # Populate the extents array with the ranges from rp structure. ranges = [] c = record_count i = 0 chunk_size = struct.calcsize('q') buf_size = len(rp_list) while c > 0 and i < buf_size: next_vcn = struct.unpack_from('q', rp_list, offset=i) lcn = struct.unpack_from('q', rp_list, offset=i + chunk_size) ranges.append((next_vcn[0], lcn[0])) i += chunk_size * 2 c -= 1 if not translate_to_extents: return ranges else: return [x for x in logical_ranges_to_extents(ranges)] # Tell Windows to make this file compressed on disk. # Only used for the test suite. def file_make_compressed(file_handle): # Assemble input structure. # Just tell Windows to use standard compression. input_struct = struct.pack('H', COMPRESSION_FORMAT_DEFAULT) buf_size = struct.calcsize('H') _ = DeviceIoControl(file_handle, FSCTL_SET_COMPRESSION, input_struct, buf_size) # Tell Windows to make this file sparse on disk. # Only used for the test suite. def file_make_sparse(file_handle): _ = DeviceIoControl(file_handle, FSCTL_SET_SPARSE, None, None) # Tell Windows to add a zero region to a sparse file. # Only used for the test suite. def file_add_sparse_region(file_handle, byte_start, byte_end): # Assemble input structure. input_struct = struct.pack('qq', byte_start, byte_end) buf_size = struct.calcsize('qq') _ = DeviceIoControl(file_handle, FSCTL_SET_ZERO_DATA, input_struct, buf_size) # Retrieve a bitmap of whether clusters on disk are free/allocated. def get_volume_bitmap(volume_handle, total_clusters): # Assemble input structure and query Windows for volume bitmap. # The input structure is the number 0 as a signed 64 bit integer. input_struct = struct.pack('q', 0) # Figure out the buffer size. Add small fudge factor to ensure success. buf_size = (total_clusters / 8) + 16 + 64 vb_struct = DeviceIoControl(volume_handle, FSCTL_GET_VOLUME_BITMAP, input_struct, buf_size) # At this point we have a FSCTL_GET_VOLUME_BITMAP (vb) structure. # Process content of the first part of structure. # Separate the volume bitmap up front, so we are not making too # many string copies of it. chunk_size = struct.calcsize('2q') volume_bitmap = vb_struct[chunk_size:] vb_struct = vb_struct[:chunk_size] starting_lcn, vb_struct = unpack_element('q', vb_struct) # 8 bytes bitmap_size, vb_struct = unpack_element('q', vb_struct) # 8 bytes # If we make the GET_VOLUME_BITMAP request with 0, # this should always come back 0. assert starting_lcn == 0 # The remaining part of the structure is the actual bitmap. return volume_bitmap, bitmap_size # Retrieve info about an NTFS volume. # We are mainly interested in the locations of the Master File Table. # This call is currently not necessary, but has been left in to address any # future need. def get_ntfs_volume_data(volume_handle): # 512 bytes will be comfortably enough to store return object. vd_struct = DeviceIoControl(volume_handle, FSCTL_GET_NTFS_VOLUME_DATA, None, 512) # At this point we have a FSCTL_GET_NTFS_VOLUME_DATA (vd) structure. # Pick out the elements from structure that are useful to us. _, vd_struct = unpack_element('q', vd_struct) # 8 bytes number_sectors, vd_struct = unpack_element('q', vd_struct) # 8 bytes total_clusters, vd_struct = unpack_element('q', vd_struct) # 8 bytes free_clusters, vd_struct = unpack_element('q', vd_struct) # 8 bytes total_reserved, vd_struct = unpack_element('q', vd_struct) # 8 bytes _, vd_struct = unpack_element('4I', vd_struct) # 4*4 bytes _, vd_struct = unpack_element('3q', vd_struct) # 3*8 bytes mft_zone_start, vd_struct = unpack_element('q', vd_struct) # 8 bytes mft_zone_end, vd_struct = unpack_element('q', vd_struct) # 8 bytes # Quick sanity check that we got something reasonable for MFT zone. assert (mft_zone_start < mft_zone_end and mft_zone_start > 0 and mft_zone_end > 0) logger.debug("MFT from %d to %d", mft_zone_start, mft_zone_end) return mft_zone_start, mft_zone_end # Poll to confirm that our clusters were freed. # Check ten times per second for a duration of seven seconds. # According to Windows Internals book, it may take several seconds # until NTFS does a checkpoint and releases the clusters. # In later versions of Windows, this seems to be instantaneous. def poll_clusters_freed(volume_handle, total_clusters, orig_extents): polling_duration_seconds = 7 attempts_per_second = 10 if not orig_extents: return True for _ in range(polling_duration_seconds * attempts_per_second): volume_bitmap, bitmap_size = get_volume_bitmap(volume_handle, total_clusters) count_free, count_allocated = check_extents( orig_extents, volume_bitmap) # Some inexact measure to determine if our clusters were freed # by the OS, knowing that another process may grab some clusters # in between our polling attempts. if count_free > count_allocated: return True Sleep(1000 / attempts_per_second) return False # Move a file (or portion of) to a new location on the disk using # the Defrag API. # This will raise an exception if a cluster was not free, # or if the call failed for whatever other reason. def move_file(volume_handle, file_handle, starting_vcn, starting_lcn, cluster_count): # Assemble input structure for our request. # We include a couple of zero ints for 64-bit alignment. input_struct = struct.pack('IIqqII', int(file_handle), 0, starting_vcn, starting_lcn, cluster_count, 0) vb_struct = DeviceIoControl(volume_handle, FSCTL_MOVE_FILE, input_struct, None) # Write zero-fill to a file. # Write_length is the number of bytes to be written. def write_zero_fill(file_handle, write_length): # Bytearray will be initialized with null bytes as part of constructor. fill_string = bytearray(write_buf_size) assert len(fill_string) == write_buf_size # Loop and perform writes of write_buf_size bytes or less. # Continue until write_length bytes have been written. # There is no need to explicitly move the file pointer while # writing. We are writing contiguously. while write_length > 0: if write_length >= write_buf_size: write_string = fill_string write_length -= write_buf_size else: write_string = fill_string[:write_length] write_length = 0 # Write buffer to file. #logger.debug("Write %d bytes", len(write_string)) _, bytes_written = WriteFile(file_handle, write_string) assert bytes_written == len(write_string) FlushFileBuffers(file_handle) # Wipe the file using the extents list we have built. # We just rewrite the file with enough zeros to cover all clusters. def wipe_file_direct(file_handle, extents, cluster_size, file_size): assert cluster_size > 0 # Remember that file_size measures full expanded content of the file, # which may not always match with size on disk (eg. if file compressed). LockFile(file_handle, 0, 0, file_size & 0xFFFF, file_size >> 16) if extents: # Use size on disk to determine how many clusters of zeros we write. for lcn_start, lcn_end in extents: # logger.debug("Wiping extent from %d to %d...", # lcn_start, lcn_end) write_length = (lcn_end - lcn_start + 1) * cluster_size write_zero_fill(file_handle, write_length) else: # Special case - file so small it can be contained within the # directory entry in the MFT part of the disk. #logger.debug("Wiping tiny file that fits entirely on MFT") write_length = file_size write_zero_fill(file_handle, write_length) # Wipe an extent by making calls to the defrag API. # We create a new zero-filled file, then move its clusters to the # position on disk that we want to wipe. # Use a look-ahead with the volume bitmap to figure out if we can expect # our call to succeed. # If not, break the extent into smaller pieces efficiently. # Windows concepts: # LCN (Logical Cluster Number) = a cluster location on disk; an absolute # position on the volume we are writing # VCN (Virtual Cluster Number) = relative position within a file, measured # in clusters def wipe_extent_by_defrag(volume_handle, lcn_start, lcn_end, cluster_size, total_clusters, tmp_file_path): assert cluster_size > 0 logger.debug("Examining extent from %d to %d for wipe...", lcn_start, lcn_end) write_length = (lcn_end - lcn_start + 1) * cluster_size # Check the state of the volume bitmap for the extent we want to # overwrite. If any sectors are allocated, reduce the task # into smaller parts. # We also reduce to smaller pieces if the extent is larger than # 2 megabytes. For no particular reason except to avoid the entire # request failing because one cluster became allocated. volume_bitmap, bitmap_size = get_volume_bitmap(volume_handle, total_clusters) # This option simulates another process that grabs clusters on disk # from time to time. # It should be moved away after QA is complete. if not simulate_concurrency: count_free, count_allocated = check_extents( [(lcn_start, lcn_end)], volume_bitmap) else: count_free, count_allocated = check_extents_concurrency( [(lcn_start, lcn_end)], volume_bitmap, tmp_file_path, volume_handle, total_clusters) if count_allocated > 0 and count_free == 0: return False if count_allocated > 0 or write_length > write_buf_size * 4: if lcn_start >= lcn_end: return False for split_s, split_e in split_extent(lcn_start, lcn_end): wipe_extent_by_defrag(volume_handle, split_s, split_e, cluster_size, total_clusters, tmp_file_path) return True # Put the zero-fill file in place. file_handle = CreateFile(tmp_file_path, GENERIC_READ | GENERIC_WRITE, 0, None, CREATE_ALWAYS, FILE_ATTRIBUTE_HIDDEN, None) write_zero_fill(file_handle, write_length) new_extents = get_extents(file_handle) # We know the original extent was contiguous. # The new zero-fill file may not be contiguous, so it requires a # loop to be sure of reaching the end of the new file's clusters. new_vcn = 0 for new_lcn_start, new_lcn_end in new_extents: # logger.debug("Zero-fill wrote from %d to %d", # new_lcn_start, new_lcn_end) cluster_count = new_lcn_end - new_lcn_start + 1 cluster_dest = lcn_start + new_vcn if new_lcn_start != cluster_dest: logger.debug("Move %d clusters to %d", cluster_count, cluster_dest) try: move_file(volume_handle, file_handle, new_vcn, cluster_dest, cluster_count) except: # Move file failed, probably because another process # has allocated a cluster on disk. # Break into smaller pieces and do what we can. logger.debug("!! Move encountered an error !!") CloseHandle(file_handle) if lcn_start >= lcn_end: return False for split_s, split_e in split_extent(lcn_start, lcn_end): wipe_extent_by_defrag(volume_handle, split_s, split_e, cluster_size, total_clusters, tmp_file_path) return True else: # If Windows put the zero-fill extent on the exact clusters we # intended to place it, no need to attempt a move. logging.debug("No need to move extent from %d", new_lcn_start) new_vcn += cluster_count CloseHandle(file_handle) DeleteFile(tmp_file_path) return True # Clean up open handles etc. def clean_up(file_handle, volume_handle, tmp_file_path): try: if file_handle: CloseHandle(file_handle) if volume_handle: CloseHandle(volume_handle) if tmp_file_path: DeleteFile(tmp_file_path) except: pass # Main flow of control. def file_wipe(file_name): # add \\?\ if it does not exist to support Unicode and long paths file_name = extended_path(file_name) check_os() win_version, _ = determine_win_version() volume = volume_from_file(file_name) volume_info = get_volume_information(volume) cluster_size = (volume_info.sectors_per_cluster * volume_info.bytes_per_sector) file_handle = open_file(file_name) file_size, is_special = get_file_basic_info(file_name, file_handle) orig_extents = get_extents(file_handle) if is_special: bridged_extents = [x for x in logical_ranges_to_extents( get_extents(file_handle, False), True)] CloseHandle(file_handle) #logger.debug('Original extents: {}'.format(orig_extents)) volume_handle = obtain_readwrite(volume) attrs = GetFileAttributesW(file_name) if attrs & FILE_ATTRIBUTE_READONLY: # Remove read-only attribute to avoid "access denied" in CreateFileW(). SetFileAttributesW(file_name, attrs & ~FILE_ATTRIBUTE_READONLY) file_handle = open_file(file_name, GENERIC_READ | GENERIC_WRITE) if not is_special: # Direct overwrite when it's a regular file. #logger.info("Attempting direct file wipe.") wipe_file_direct(file_handle, orig_extents, cluster_size, file_size) new_extents = get_extents(file_handle) CloseHandle(file_handle) #logger.debug('New extents: {}'.format(new_extents)) if orig_extents == new_extents: clean_up(None, volume_handle, None) return # Expectation was that extents should be identical and file is wiped. # If OS didn't give that to us, continue below and use defrag wipe. # Any extent within new_extents has now been wiped by above. # It can be subtracted from the orig_extents list, and now we will # just clean up anything not yet overwritten. orig_extents = extents_a_minus_b(orig_extents, new_extents) else: # File needs special treatment. We can't just do a basic overwrite. # First we will truncate it. Then chase down the freed clusters to # wipe them, now that they are no longer part of the file. truncate_file(file_handle) CloseHandle(file_handle) # Poll to confirm that our clusters were freed. poll_clusters_freed(volume_handle, volume_info.total_clusters, orig_extents) # Chase down all the freed clusters we can, and wipe them. #logger.debug("Attempting defrag file wipe.") # Put the temp file in the same folder as the target wipe file. # Should be able to write this path if user can write the wipe file. tmp_file_path = os.path.dirname(file_name) + os.sep + tmp_file_name if is_special: orig_extents = choose_if_bridged(volume_handle, volume_info.total_clusters, orig_extents, bridged_extents) for lcn_start, lcn_end in orig_extents: result = wipe_extent_by_defrag(volume_handle, lcn_start, lcn_end, cluster_size, volume_info.total_clusters, tmp_file_path) # Clean up. clean_up(None, volume_handle, tmp_file_path) return bleachbit-4.4.2/bleachbit/Worker.py0000775000175000017500000003344414144024253016134 0ustar fabiofabio# vim: ts=4:sw=4:expandtab # BleachBit # Copyright (C) 2008-2021 Andrew Ziem # https://www.bleachbit.org # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . """ Perform the preview or delete operations """ from bleachbit import DeepScan, FileUtilities from bleachbit.Cleaner import backends from bleachbit import _, ngettext import logging import math import sys import os logger = logging.getLogger(__name__) class Worker: """Perform the preview or delete operations""" def __init__(self, ui, really_delete, operations): """Create a Worker ui: an instance with methods append_text() update_progress_bar() update_total_size() update_item_size() worker_done() really_delete: (boolean) preview or make real changes? operations: dictionary where operation-id is the key and operation-id are values """ self.ui = ui self.really_delete = really_delete assert(isinstance(operations, dict)) self.operations = operations self.size = 0 self.total_bytes = 0 self.total_deleted = 0 self.total_errors = 0 self.total_special = 0 # special operations self.yield_time = None self.is_aborted = False if 0 == len(self.operations): raise RuntimeError("No work to do") def abort(self): """Stop the preview/cleaning operation""" self.is_aborted = True def print_exception(self, operation): """Display exception""" # TRANSLATORS: This indicates an error. The special keyword # %(operation)s will be replaced by 'firefox' or 'opera' or # some other cleaner ID. The special keyword %(msg)s will be # replaced by a message such as 'Permission denied.' err = _("Exception while running operation '%(operation)s': '%(msg)s'") \ % {'operation': operation, 'msg': str(sys.exc_info()[1])} logger.error(err, exc_info=True) self.total_errors += 1 def execute(self, cmd, operation_option): """Execute or preview the command""" ret = None try: for ret in cmd.execute(self.really_delete): if True == ret or isinstance(ret, tuple): # Temporarily pass control to the GTK idle loop, # allow user to abort, and # display progress (if applicable). yield ret if self.is_aborted: return except SystemExit: pass except Exception as e: # 2 = does not exist # 13 = permission denied from errno import ENOENT, EACCES if isinstance(e, OSError) and e.errno in (ENOENT, EACCES): # For access denied, do not show traceback exc_message = str(e) logger.error('%s: %s', exc_message, cmd) else: # For other errors, show the traceback. msg = _('Error: {operation_option}: {command}') data = {'command': cmd, 'operation_option': operation_option} logger.error(msg.format(**data), exc_info=True) self.total_errors += 1 else: if ret is None: return if isinstance(ret['size'], int): size = FileUtilities.bytes_to_human(ret['size']) self.size += ret['size'] self.total_bytes += ret['size'] else: size = "?B" if ret['path']: path = ret['path'] else: path = '' line = "%s %s %s\n" % (ret['label'], size, path) self.total_deleted += ret['n_deleted'] self.total_special += ret['n_special'] if ret['label']: # the label may be a hidden operation # (e.g., win.shell.change.notify) self.ui.append_text(line) def clean_operation(self, operation): """Perform a single cleaning operation""" operation_options = self.operations[operation] assert(isinstance(operation_options, list)) logger.debug("clean_operation('%s'), options = '%s'", operation, operation_options) if not operation_options: return if self.really_delete and backends[operation].is_running(): # TRANSLATORS: %s expands to a name such as 'Firefox' or 'System'. err = _("%s cannot be cleaned because it is currently running. Close it, and try again.") \ % backends[operation].get_name() self.ui.append_text(err + "\n", 'error') self.total_errors += 1 return import time self.yield_time = time.time() total_size = 0 for option_id in operation_options: self.size = 0 assert(isinstance(option_id, str)) # normal scan for cmd in backends[operation].get_commands(option_id): for ret in self.execute(cmd, '%s.%s' % (operation, option_id)): if True == ret: # Return control to PyGTK idle loop to keep # it responding allow the user to abort self.yield_time = time.time() yield True if self.is_aborted: break if time.time() - self.yield_time > 0.25: if self.really_delete: self.ui.update_total_size(self.total_bytes) yield True self.yield_time = time.time() self.ui.update_item_size(operation, option_id, self.size) total_size += self.size # deep scan for (path, search) in backends[operation].get_deep_scan(option_id): if '' == path: path = os.path.expanduser('~') if search.command not in ('delete', 'shred'): raise NotImplementedError( 'Deep scan only supports deleting or shredding now') if path not in self.deepscans: self.deepscans[path] = [] self.deepscans[path].append(search) self.ui.update_item_size(operation, -1, total_size) def run_delayed_op(self, operation, option_id): """Run one delayed operation""" self.ui.update_progress_bar(0.0) if 'free_disk_space' == option_id: # TRANSLATORS: 'free' means 'unallocated' msg = _("Please wait. Wiping free disk space.") self.ui.append_text( _('Wiping free disk space erases remnants of files that were deleted without shredding. It does not free up space.')) self.ui.append_text('\n') elif 'memory' == option_id: msg = _("Please wait. Cleaning %s.") % _("Memory") else: raise RuntimeError("Unexpected option_id in delayed ops") self.ui.update_progress_bar(msg) for cmd in backends[operation].get_commands(option_id): for ret in self.execute(cmd, '%s.%s' % (operation, option_id)): if isinstance(ret, tuple): # Display progress (for free disk space) phase = ret[0] # A while ago there were other phase numbers. Currently it's just 1 if phase != 1: raise RuntimeError( 'While wiping free space, unexpected phase %d' % phase) percent_done = ret[1] eta_seconds = ret[2] self.ui.update_progress_bar(percent_done) if isinstance(eta_seconds, int): eta_mins = math.ceil(eta_seconds / 60) msg2 = ngettext("About %d minute remaining.", "About %d minutes remaining.", eta_mins) \ % eta_mins self.ui.update_progress_bar(msg + ' ' + msg2) else: self.ui.update_progress_bar(msg) if self.is_aborted: break if True == ret or isinstance(ret, tuple): # Return control to PyGTK idle loop to keep # it responding and allow the user to abort. yield True def run(self): """Perform the main cleaning process which has these phases 1. General cleaning 2. Deep scan 3. Memory 4. Free disk space""" self.deepscans = {} # prioritize self.delayed_ops = [] for operation in self.operations: delayables = ['free_disk_space', 'memory'] for delayable in delayables: if operation not in ('system', '_gui'): continue if delayable in self.operations[operation]: i = self.operations[operation].index(delayable) del self.operations[operation][i] priority = 99 if 'free_disk_space' == delayable: priority = 100 new_op = (priority, {operation: [delayable]}) self.delayed_ops.append(new_op) # standard operations import warnings with warnings.catch_warnings(record=True) as ws: # This warning system allows general warnings. Duplicate will # be removed, and the warnings will show near the end of # the log. warnings.simplefilter('once') for _dummy in self.run_operations(self.operations): # yield to GTK+ idle loop yield True for w in ws: logger.warning(w.message) # run deep scan if self.deepscans: yield from self.run_deep_scan() # delayed operations for op in sorted(self.delayed_ops): operation = list(op[1].keys())[0] for option_id in list(op[1].values())[0]: for _ret in self.run_delayed_op(operation, option_id): # yield to GTK+ idle loop yield True # print final stats bytes_delete = FileUtilities.bytes_to_human(self.total_bytes) if self.really_delete: # TRANSLATORS: This refers to disk space that was # really recovered (in other words, not a preview) line = _("Disk space recovered: %s") % bytes_delete else: # TRANSLATORS: This refers to a preview (no real # changes were made yet) line = _("Disk space to be recovered: %s") % bytes_delete self.ui.append_text("\n%s" % line) if self.really_delete: # TRANSLATORS: This refers to the number of files really # deleted (in other words, not a preview). line = _("Files deleted: %d") % self.total_deleted else: # TRANSLATORS: This refers to the number of files that # would be deleted (in other words, simply a preview). line = _("Files to be deleted: %d") % self.total_deleted self.ui.append_text("\n%s" % line) if self.total_special > 0: line = _("Special operations: %d") % self.total_special self.ui.append_text("\n%s" % line) if self.total_errors > 0: line = _("Errors: %d") % self.total_errors self.ui.append_text("\n%s" % line, 'error') self.ui.append_text('\n') if self.really_delete: self.ui.update_total_size(self.total_bytes) self.ui.worker_done(self, self.really_delete) yield False def run_deep_scan(self): """Run deep scans""" logger.debug(' deepscans=%s' % self.deepscans) # TRANSLATORS: The "deep scan" feature searches over broad # areas of the file system such as the user's whole home directory # or all the system executables. self.ui.update_progress_bar(_("Please wait. Running deep scan.")) yield True # allow GTK to update the screen ds = DeepScan.DeepScan(self.deepscans) for cmd in ds.scan(): if True == cmd: yield True continue for _ret in self.execute(cmd, 'deepscan'): yield True def run_operations(self, my_operations): """Run a set of operations (general, memory, free disk space)""" for count, operation in enumerate(my_operations): self.ui.update_progress_bar(1.0 * count / len(my_operations)) name = backends[operation].get_name() if self.really_delete: # TRANSLATORS: %s is replaced with Firefox, System, etc. msg = _("Please wait. Cleaning %s.") % name else: # TRANSLATORS: %s is replaced with Firefox, System, etc. msg = _("Please wait. Previewing %s.") % name self.ui.update_progress_bar(msg) yield True # show the progress bar message now try: for _dummy in self.clean_operation(operation): yield True except: self.print_exception(operation) bleachbit-4.4.2/bleachbit/Command.py0000775000175000017500000002456614144024253016246 0ustar fabiofabio# vim: ts=4:sw=4:expandtab # BleachBit # Copyright (C) 2008-2021 Andrew Ziem # https://www.bleachbit.org # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . """ Command design pattern implementation for cleaning """ from bleachbit import _ from bleachbit import FileUtilities import logging import os import types from sqlite3 import DatabaseError if 'nt' == os.name: import bleachbit.Windows else: from bleachbit.General import WindowsError def whitelist(path): """Return information that this file was whitelisted""" ret = { # TRANSLATORS: This is the label in the log indicating was # skipped because it matches the whitelist 'label': _('Skip'), 'n_deleted': 0, 'n_special': 0, 'path': path, 'size': 0} return ret class Delete: """Delete a single file or directory. Obey the user preference regarding shredding.""" def __init__(self, path): """Create a Delete instance to delete 'path'""" self.path = path self.shred = False def __str__(self): return 'Command to %s %s' % \ ('shred' if self.shred else 'delete', self.path) def execute(self, really_delete): """Make changes and return results""" if FileUtilities.whitelisted(self.path): yield whitelist(self.path) return ret = { # TRANSLATORS: This is the label in the log indicating will be # deleted (for previews) or was actually deleted 'label': _('Delete'), 'n_deleted': 1, 'n_special': 0, 'path': self.path, 'size': FileUtilities.getsize(self.path)} if really_delete: try: FileUtilities.delete(self.path, self.shred) except WindowsError as e: # WindowsError: [Error 32] The process cannot access the file because it is being # used by another process: 'C:\\Documents and # Settings\\username\\Cookies\\index.dat' if 32 != e.winerror and 5 != e.winerror: raise try: bleachbit.Windows.delete_locked_file(self.path) except: raise else: if self.shred: import warnings warnings.warn( _('At least one file was locked by another process, so its contents could not be overwritten. It will be marked for deletion upon system reboot.')) # TRANSLATORS: The file will be deleted when the # system reboots ret['label'] = _('Mark for deletion') yield ret class Function: """Execute a simple Python function""" def __init__(self, path, func, label): """Path is a pathname that exists or None. If it exists, func takes the pathname. Otherwise, function returns the size.""" self.path = path self.func = func self.label = label try: assert isinstance(func, types.FunctionType) except AssertionError: raise AssertionError('Expected MethodType but got %s' % type(func)) def __str__(self): if self.path: return 'Function: %s: %s' % (self.label, self.path) else: return 'Function: %s' % (self.label) def execute(self, really_delete): if self.path is not None and FileUtilities.whitelisted(self.path): yield whitelist(self.path) return ret = { 'label': self.label, 'n_deleted': 0, 'n_special': 1, 'path': self.path, 'size': None} if really_delete: if self.path is None: # Function takes no path. It returns the size. func_ret = self.func() if isinstance(func_ret, types.GeneratorType): # function returned generator for func_ret in self.func(): if True == func_ret or isinstance(func_ret, tuple): # Return control to GTK idle loop. # If tuple, then display progress. yield func_ret # either way, func_ret should be an integer assert isinstance(func_ret, int) ret['size'] = func_ret else: if os.path.isdir(self.path): raise RuntimeError('Attempting to run file function %s on directory %s' % (self.func.__name__, self.path)) # Function takes a path. We check the size. oldsize = FileUtilities.getsize(self.path) try: self.func(self.path) except DatabaseError as e: if -1 == e.message.find('file is encrypted or is not a database') and \ -1 == e.message.find('or missing database'): raise logging.getLogger(__name__).exception(e.message) return try: newsize = FileUtilities.getsize(self.path) except OSError as e: from errno import ENOENT if e.errno == ENOENT: # file does not exist newsize = 0 else: raise ret['size'] = oldsize - newsize yield ret class Ini: """Remove sections or parameters from a .ini file""" def __init__(self, path, section, parameter): """Create the instance""" self.path = path self.section = section self.parameter = parameter def __str__(self): return 'Command to clean .ini path=%s, section=%s, parameter=%s ' % \ (self.path, self.section, self.parameter) def execute(self, really_delete): """Make changes and return results""" if FileUtilities.whitelisted(self.path): yield whitelist(self.path) return ret = { # TRANSLATORS: Parts of this file will be deleted 'label': _('Clean file'), 'n_deleted': 0, 'n_special': 1, 'path': self.path, 'size': None} if really_delete: oldsize = FileUtilities.getsize(self.path) FileUtilities.clean_ini(self.path, self.section, self.parameter) newsize = FileUtilities.getsize(self.path) ret['size'] = oldsize - newsize yield ret class Json: """Remove a key from a JSON configuration file""" def __init__(self, path, address): """Create the instance""" self.path = path self.address = address def __str__(self): return 'Command to clean JSON file, path=%s, address=%s ' % \ (self.path, self.address) def execute(self, really_delete): """Make changes and return results""" if FileUtilities.whitelisted(self.path): yield whitelist(self.path) return ret = { 'label': _('Clean file'), 'n_deleted': 0, 'n_special': 1, 'path': self.path, 'size': None} if really_delete: oldsize = FileUtilities.getsize(self.path) FileUtilities.clean_json(self.path, self.address) newsize = FileUtilities.getsize(self.path) ret['size'] = oldsize - newsize yield ret class Shred(Delete): """Shred a single file""" def __init__(self, path): """Create an instance to shred 'path'""" Delete.__init__(self, path) self.shred = True def __str__(self): return 'Command to shred %s' % self.path class Truncate(Delete): """Truncate a single file""" def __str__(self): return 'Command to truncate %s' % self.path def execute(self, really_delete): """Make changes and return results""" if FileUtilities.whitelisted(self.path): yield whitelist(self.path) return ret = { # TRANSLATORS: The file will be truncated to 0 bytes in length 'label': _('Truncate'), 'n_deleted': 1, 'n_special': 0, 'path': self.path, 'size': FileUtilities.getsize(self.path)} if really_delete: with open(self.path, 'w') as f: f.truncate(0) yield ret class Winreg: """Clean Windows registry""" def __init__(self, keyname, valuename): """Create the Windows registry cleaner""" self.keyname = keyname self.valuename = valuename def __str__(self): return 'Command to clean registry, key=%s, value=%s ' % (self.keyname, self.valuename) def execute(self, really_delete): """Execute the Windows registry cleaner""" if 'nt' != os.name: return _str = None # string representation ret = None # return value meaning 'deleted' or 'delete-able' if self.valuename: _str = '%s<%s>' % (self.keyname, self.valuename) ret = bleachbit.Windows.delete_registry_value(self.keyname, self.valuename, really_delete) else: ret = bleachbit.Windows.delete_registry_key( self.keyname, really_delete) _str = self.keyname if not ret: # Nothing to delete or nothing was deleted. This return # makes the auto-hide feature work nicely. return ret = { 'label': _('Delete registry key'), 'n_deleted': 0, 'n_special': 1, 'path': _str, 'size': 0} yield ret bleachbit-4.4.2/bleachbit/CLI.py0000775000175000017500000002521314144024253015265 0ustar fabiofabio#!/usr/bin/python3 # vim: ts=4:sw=4:expandtab # BleachBit # Copyright (C) 2008-2021 Andrew Ziem # https://www.bleachbit.org # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . """ Command line interface """ from bleachbit.Cleaner import backends, create_simple_cleaner, register_cleaners from bleachbit import _, APP_VERSION from bleachbit import SystemInformation, Options, Worker import logging import optparse import os import sys logger = logging.getLogger(__name__) class CliCallback: """Command line's callback passed to Worker""" def append_text(self, msg, tag=None): """Write text to the terminal""" print(msg.strip('\n')) def update_progress_bar(self, status): """Not used""" pass def update_total_size(self, size): """Not used""" pass def update_item_size(self, op, opid, size): """Not used""" pass def worker_done(self, worker, really_delete): """Not used""" pass def cleaners_list(): """Yield each cleaner-option pair""" list(register_cleaners()) for key in sorted(backends): c_id = backends[key].get_id() for (o_id, _o_name) in backends[key].get_options(): yield "%s.%s" % (c_id, o_id) def list_cleaners(): """Display available cleaners""" for cleaner in cleaners_list(): print (cleaner) def preview_or_clean(operations, really_clean): """Preview deletes and other changes""" cb = CliCallback() worker = Worker.Worker(cb, really_clean, operations).run() while next(worker): pass def args_to_operations_list(preset, all_but_warning): """For --preset and --all-but-warning return list of operations as list Example return: ['google_chrome.cache', 'system.tmp'] """ args = [] if not backends: list(register_cleaners()) assert(len(backends) > 1) for key in sorted(backends): c_id = backends[key].get_id() for (o_id, _o_name) in backends[key].get_options(): # restore presets from the GUI if preset and Options.options.get_tree(c_id, o_id): args.append('.'.join([c_id, o_id])) elif all_but_warning and not backends[c_id].get_warning(o_id): args.append('.'.join([c_id, o_id])) return args def args_to_operations(args, preset, all_but_warning): """Read arguments and return list of operations as dictionary""" list(register_cleaners()) operations = {} if not args: args = [] args = set(args + args_to_operations_list(preset, all_but_warning)) for arg in args: if 2 != len(arg.split('.')): logger.warning(_("not a valid cleaner: %s"), arg) continue (cleaner_id, option_id) = arg.split('.') # enable all options (for example, firefox.*) if '*' == option_id: if cleaner_id in operations: del operations[cleaner_id] operations[cleaner_id] = [ option_id2 for (option_id2, _o_name) in backends[cleaner_id].get_options() ] continue # add the specified option if cleaner_id not in operations: operations[cleaner_id] = [] if option_id not in operations[cleaner_id]: operations[cleaner_id].append(option_id) for (k, v) in operations.items(): operations[k] = sorted(v) return operations def process_cmd_line(): """Parse the command line and execute given commands.""" # TRANSLATORS: This is the command line usage. Don't translate # %prog, but do translate options, cleaner, and option. # Don't translate and add "usage:" - it gets added by Python. # More information about the command line is here # https://www.bleachbit.org/documentation/command-line usage = _("usage: %prog [options] cleaner.option1 cleaner.option2") parser = optparse.OptionParser(usage) parser.add_option("-l", "--list-cleaners", action="store_true", help=_("list cleaners")) parser.add_option("-c", "--clean", action="store_true", # TRANSLATORS: predefined cleaners are for applications, such as Firefox and Flash. # This is different than cleaning an arbitrary file, such as a # spreadsheet on the desktop. help=_("run cleaners to delete files and make other permanent changes")) parser.add_option( '--debug', help=_("set log level to verbose"), action="store_true") parser.add_option('--debug-log', help=_("log debug messages to file")) parser.add_option("-s", "--shred", action="store_true", help=_("shred specific files or folders")) parser.add_option("--sysinfo", action="store_true", help=_("show system information")) parser.add_option("--gui", action="store_true", help=_("launch the graphical interface")) parser.add_option('--exit', action='store_true', help=optparse.SUPPRESS_HELP) if 'nt' == os.name: uac_help = _("do not prompt for administrator privileges") else: uac_help = optparse.SUPPRESS_HELP parser.add_option("--no-uac", action="store_true", help=uac_help) parser.add_option("-p", "--preview", action="store_true", help=_("preview files to be deleted and other changes")) parser.add_option('--pot', action='store_true', help=optparse.SUPPRESS_HELP) parser.add_option("--preset", action="store_true", help=_("use options set in the graphical interface")) parser.add_option("--all-but-warning", action="store_true", help=_("enable all options that do not have a warning")) if 'nt' == os.name: parser.add_option("--update-winapp2", action="store_true", help=_("update winapp2.ini, if a new version is available")) parser.add_option("-w", "--wipe-free-space", action="store_true", help=_("wipe free space in the given paths")) parser.add_option("-v", "--version", action="store_true", help=_("output version information and exit")) parser.add_option('-o', '--overwrite', action='store_true', help=_('overwrite files to hide contents')) def expand_context_menu_option(option, opt, value, parser): setattr(parser.values, 'gui', True) setattr(parser.values, 'exit', True) parser.add_option("--context-menu", action="callback", callback=expand_context_menu_option) (options, args) = parser.parse_args() cmd_list = (options.list_cleaners, options.wipe_free_space, options.preview, options.clean) cmd_count = sum(x is True for x in cmd_list) if cmd_count > 1: logger.error( _('Specify only one of these commands: --list-cleaners, --wipe-free-space, --preview, --clean')) sys.exit(1) did_something = False if options.debug: # set in __init__ so it takes effect earlier pass if options.debug_log: logger.addHandler(logging.FileHandler(options.debug_log)) logger.info(SystemInformation.get_system_information()) if options.version: print(""" BleachBit version %s Copyright (C) 2008-2021 Andrew Ziem. All rights reserved. License GPLv3+: GNU GPL version 3 or later . This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.""" % APP_VERSION) sys.exit(0) if 'nt' == os.name and options.update_winapp2: from bleachbit import Update logger.info(_("Checking online for updates to winapp2.ini")) Update.check_updates(False, True, lambda x: sys.stdout.write("%s\n" % x), lambda: None) # updates can be combined with --list-cleaners, --preview, --clean did_something = True if options.list_cleaners: list_cleaners() sys.exit(0) if options.pot: from bleachbit.CleanerML import create_pot create_pot() sys.exit(0) if options.wipe_free_space: if len(args) < 1: logger.error(_("No directories given for --wipe-free-space")) sys.exit(1) for wipe_path in args: if not os.path.isdir(wipe_path): logger.error( _("Path to wipe must be an existing directory: %s"), wipe_path) sys.exit(1) logger.info(_("Wiping free space can take a long time.")) for wipe_path in args: logger.info('Wiping free space in path: %s', wipe_path) import bleachbit.FileUtilities for _ret in bleachbit.FileUtilities.wipe_path(wipe_path): pass sys.exit(0) if options.preview or options.clean: operations = args_to_operations( args, options.preset, options.all_but_warning) if not operations: logger.error(_("No work to do. Specify options.")) sys.exit(1) if options.preview: preview_or_clean(operations, False) sys.exit(0) if options.overwrite: if not options.clean or options.shred: logger.warning( _("--overwrite is intended only for use with --clean")) Options.options.set('shred', True, commit=False) if options.clean: preview_or_clean(operations, True) sys.exit(0) if options.gui: import bleachbit.GUI app = bleachbit.GUI.Bleachbit( uac=not options.no_uac, shred_paths=args, auto_exit=options.exit) sys.exit(app.run()) if options.shred: # delete arbitrary files without GUI # create a temporary cleaner object backends['_gui'] = create_simple_cleaner(args) operations = {'_gui': ['files']} preview_or_clean(operations, True) sys.exit(0) if options.sysinfo: print(SystemInformation.get_system_information()) sys.exit(0) if not did_something: parser.print_help() if __name__ == '__main__': process_cmd_line() bleachbit-4.4.2/bleachbit/Special.py0000775000175000017500000003564414144024253016247 0ustar fabiofabio# vim: ts=4:sw=4:expandtab # BleachBit # Copyright (C) 2008-2021 Andrew Ziem # https://www.bleachbit.org # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . """ Cross-platform, special cleaning operations """ from bleachbit.Options import options from bleachbit import FileUtilities import logging import os.path logger = logging.getLogger(__name__) def __get_chrome_history(path, fn='History'): """Get Google Chrome or Chromium history version. 'path' is name of any file in same directory""" path_history = os.path.join(os.path.dirname(path), fn) ver = get_sqlite_int( path_history, 'select value from meta where key="version"')[0] assert ver > 1 return ver def __shred_sqlite_char_columns(table, cols=None, where=""): """Create an SQL command to shred character columns""" cmd = "" if not where: # If None, set to empty string. where = "" if cols and options.get('shred'): cmd += "update or ignore %s set %s %s;" % \ (table, ",".join(["%s = randomblob(length(%s))" % (col, col) for col in cols]), where) cmd += "update or ignore %s set %s %s;" % \ (table, ",".join(["%s = zeroblob(length(%s))" % (col, col) for col in cols]), where) cmd += "delete from %s %s;" % (table, where) return cmd def __sqlite_table_exists(pathname, table): """Check whether a table exists in the SQLite database""" cmd = "select name from sqlite_master where type='table' and name=?;" import sqlite3 conn = sqlite3.connect(pathname) cursor = conn.cursor() ret = False cursor.execute(cmd, (table,)) if cursor.fetchone(): ret = True cursor.close() conn.commit() conn.close() return ret def get_sqlite_int(path, sql, parameters=None): """Run SQL on database in 'path' and return the integers""" import sqlite3 conn = sqlite3.connect(path) cursor = conn.cursor() if parameters: cursor.execute(sql, parameters) else: cursor.execute(sql) ids = [int(row[0]) for row in cursor] cursor.close() conn.close() return ids def delete_chrome_autofill(path): """Delete autofill table in Chromium/Google Chrome 'Web Data' database""" cols = ('name', 'value', 'value_lower') cmds = __shred_sqlite_char_columns('autofill', cols) cols = ('first_name', 'middle_name', 'last_name', 'full_name') cmds += __shred_sqlite_char_columns('autofill_profile_names', cols) cmds += __shred_sqlite_char_columns('autofill_profile_emails', ('email',)) cmds += __shred_sqlite_char_columns('autofill_profile_phones', ('number',)) cols = ('company_name', 'street_address', 'dependent_locality', 'city', 'state', 'zipcode', 'country_code') cmds += __shred_sqlite_char_columns('autofill_profiles', cols) cols = ( 'company_name', 'street_address', 'address_1', 'address_2', 'address_3', 'address_4', 'postal_code', 'country_code', 'language_code', 'recipient_name', 'phone_number') cmds += __shred_sqlite_char_columns('server_addresses', cols) FileUtilities.execute_sqlite3(path, cmds) def delete_chrome_databases_db(path): """Delete remote HTML5 cookies (avoiding extension data) from the Databases.db file""" cols = ('origin', 'name', 'description') where = "where origin not like 'chrome-%'" cmds = __shred_sqlite_char_columns('Databases', cols, where) FileUtilities.execute_sqlite3(path, cmds) def delete_chrome_favicons(path): """Delete Google Chrome and Chromium favicons not use in in history for bookmarks""" path_history = os.path.join(os.path.dirname(path), 'History') if os.path.exists(path_history): ver = __get_chrome_history(path) else: # assume it's the newer version ver = 38 cmds = "" if ver >= 4: # Version 4 includes Chromium 12 # Version 20 includes Chromium 14, Google Chrome 15, Google Chrome 19 # Version 22 includes Google Chrome 20 # Version 25 is Google Chrome 26 # Version 26 is Google Chrome 29 # Version 28 is Google Chrome 30 # Version 29 is Google Chrome 37 # Version 32 is Google Chrome 51 # Version 36 is Google Chrome 60 # Version 38 is Google Chrome 64 # Version 42 is Google Chrome 79 # icon_mapping cols = ('page_url',) where = None if os.path.exists(path_history): cmds += "attach database \"%s\" as History;" % path_history where = "where page_url not in (select distinct url from History.urls)" cmds += __shred_sqlite_char_columns('icon_mapping', cols, where) # favicon images cols = ('image_data', ) where = "where icon_id not in (select distinct icon_id from icon_mapping)" cmds += __shred_sqlite_char_columns('favicon_bitmaps', cols, where) # favicons # Google Chrome 30 (database version 28): image_data moved to table # favicon_bitmaps if ver < 28: cols = ('url', 'image_data') else: cols = ('url', ) where = "where id not in (select distinct icon_id from icon_mapping)" cmds += __shred_sqlite_char_columns('favicons', cols, where) elif 3 == ver: # Version 3 includes Google Chrome 11 cols = ('url', 'image_data') where = None if os.path.exists(path_history): cmds += "attach database \"%s\" as History;" % path_history where = "where id not in(select distinct favicon_id from History.urls)" cmds += __shred_sqlite_char_columns('favicons', cols, where) else: raise RuntimeError('%s is version %d' % (path, ver)) FileUtilities.execute_sqlite3(path, cmds) def delete_chrome_history(path): """Clean history from History and Favicon files without affecting bookmarks""" if not os.path.exists(path): logger.debug( 'aborting delete_chrome_history() because history does not exist: %s' % path) return cols = ('url', 'title') where = "" ids_int = get_chrome_bookmark_ids(path) if ids_int: ids_str = ",".join([str(id0) for id0 in ids_int]) where = "where id not in (%s) " % ids_str cmds = __shred_sqlite_char_columns('urls', cols, where) cmds += __shred_sqlite_char_columns('visits') # Google Chrome 79 no longer has lower_term in keyword_search_terms cols = ('term',) cmds += __shred_sqlite_char_columns('keyword_search_terms', cols) ver = __get_chrome_history(path) if ver >= 20: # downloads, segments, segment_usage first seen in Chrome 14, # Google Chrome 15 (database version = 20). # Google Chrome 30 (database version 28) doesn't have full_path, but it # does have current_path and target_path if ver >= 28: cmds += __shred_sqlite_char_columns( 'downloads', ('current_path', 'target_path')) cmds += __shred_sqlite_char_columns( 'downloads_url_chains', ('url', )) else: cmds += __shred_sqlite_char_columns( 'downloads', ('full_path', 'url')) cmds += __shred_sqlite_char_columns('segments', ('name',)) cmds += __shred_sqlite_char_columns('segment_usage') FileUtilities.execute_sqlite3(path, cmds) def delete_chrome_keywords(path): """Delete keywords table in Chromium/Google Chrome 'Web Data' database""" cols = ('short_name', 'keyword', 'favicon_url', 'originating_url', 'suggest_url') where = "where not date_created = 0" cmds = __shred_sqlite_char_columns('keywords', cols, where) cmds += "update keywords set usage_count = 0;" ver = __get_chrome_history(path, 'Web Data') if 43 <= ver < 49: # keywords_backup table first seen in Google Chrome 17 / Chromium 17 which is Web Data version 43 # In Google Chrome 25, the table is gone. cmds += __shred_sqlite_char_columns('keywords_backup', cols, where) cmds += "update keywords_backup set usage_count = 0;" FileUtilities.execute_sqlite3(path, cmds) def delete_office_registrymodifications(path): """Erase LibreOffice 3.4 and Apache OpenOffice.org 3.4 MRU in registrymodifications.xcu""" import xml.dom.minidom dom1 = xml.dom.minidom.parse(path) modified = False for node in dom1.getElementsByTagName("item"): if not node.hasAttribute("oor:path"): continue if not node.getAttribute("oor:path").startswith('/org.openoffice.Office.Histories/Histories/'): continue node.parentNode.removeChild(node) node.unlink() modified = True if modified: with open(path, 'w', encoding='utf-8') as xml_file: dom1.writexml(xml_file) def delete_mozilla_url_history(path): """Delete URL history in Mozilla places.sqlite (Firefox 3 and family)""" cmds = "" # delete the URLs in moz_places places_suffix = "where id in (select " \ "moz_places.id from moz_places " \ "left join moz_bookmarks on moz_bookmarks.fk = moz_places.id " \ "where moz_bookmarks.id is null); " cols = ('url', 'rev_host', 'title') cmds += __shred_sqlite_char_columns('moz_places', cols, places_suffix) # For any bookmarks that remain in moz_places, reset the non-character values. cmds += "update moz_places set visit_count=0, frecency=-1, last_visit_date=null;" # delete any orphaned annotations in moz_annos annos_suffix = "where id in (select moz_annos.id " \ "from moz_annos " \ "left join moz_places " \ "on moz_annos.place_id = moz_places.id " \ "where moz_places.id is null); " cmds += __shred_sqlite_char_columns( 'moz_annos', ('content', ), annos_suffix) # Delete any orphaned favicons. # Firefox 78 no longer has a table named moz_favicons, and it no longer has # a column favicon_id in the table moz_places. (This change probably happened before version 78.) if __sqlite_table_exists(path, 'moz_favicons'): fav_suffix = "where id not in (select favicon_id " \ "from moz_places where favicon_id is not null ); " cols = ('url', 'data') cmds += __shred_sqlite_char_columns('moz_favicons', cols, fav_suffix) # Delete orphaned origins. if __sqlite_table_exists(path, 'moz_origins'): origins_where = 'where id not in (select distinct origin_id from moz_places)' cmds += __shred_sqlite_char_columns('moz_origins', ('host',), origins_where) # For any remaining origins, reset the statistic. cmds += "update moz_origins set frecency=-1;" if __sqlite_table_exists(path, 'moz_meta'): cmds += "delete from moz_meta where key like 'origin_frecency_%';" # Delete all history visits. cmds += "delete from moz_historyvisits;" # delete any orphaned input history input_suffix = "where place_id not in (select distinct id from moz_places)" cols = ('input',) cmds += __shred_sqlite_char_columns('moz_inputhistory', cols, input_suffix) # delete the whole moz_hosts table # Reference: https://bugzilla.mozilla.org/show_bug.cgi?id=932036 # Reference: # https://support.mozilla.org/en-US/questions/937290#answer-400987 if __sqlite_table_exists(path, 'moz_hosts'): cmds += __shred_sqlite_char_columns('moz_hosts', ('host',)) cmds += "delete from moz_hosts;" # execute the commands FileUtilities.execute_sqlite3(path, cmds) def delete_mozilla_favicons(path): """Delete favorites icon in Mozilla places.favicons only if they are not bookmarks (Firefox 3 and family)""" cmds = "" places_path = os.path.join(os.path.dirname(path), 'places.sqlite') cmds += "attach database \"%s\" as places;" % places_path # delete all not bookmarked icon urls urls_where = ( "where page_url not in (select url from places.moz_places where id in " "(select distinct fk from places.moz_bookmarks where fk is not null))" ) cmds += __shred_sqlite_char_columns('moz_pages_w_icons', ('page_url',), urls_where) # delete all not bookmarked icons to pages mapping mapping_where = "where page_id not in (select id from moz_pages_w_icons)" cmds += __shred_sqlite_char_columns('moz_icons_to_pages', where=mapping_where) # delete all not bookmarked icons icons_where = "where (id not in (select icon_id from moz_icons_to_pages))" cols = ('icon_url', 'data') cmds += __shred_sqlite_char_columns('moz_icons', cols, where=icons_where) FileUtilities.execute_sqlite3(path, cmds) def delete_ooo_history(path): """Erase the OpenOffice.org MRU in Common.xcu. No longer valid in Apache OpenOffice.org 3.4.""" import xml.dom.minidom dom1 = xml.dom.minidom.parse(path) changed = False for node in dom1.getElementsByTagName("node"): if node.hasAttribute("oor:name"): if "History" == node.getAttribute("oor:name"): node.parentNode.removeChild(node) node.unlink() changed = True break if changed: dom1.writexml(open(path, "w", encoding='utf-8')) def get_chrome_bookmark_ids(history_path): """Given the path of a history file, return the ids in the urls table that are bookmarks""" bookmark_path = os.path.join(os.path.dirname(history_path), 'Bookmarks') if not os.path.exists(bookmark_path): return [] urls = get_chrome_bookmark_urls(bookmark_path) ids = [] for url in urls: ids += get_sqlite_int( history_path, 'select id from urls where url=?', (url,)) return ids def get_chrome_bookmark_urls(path): """Return a list of bookmarked URLs in Google Chrome/Chromium""" import json # read file to parser with open(path, 'r', encoding='utf-8') as f: js = json.load(f) # empty list urls = [] # local recursive function def get_chrome_bookmark_urls_helper(node): if not isinstance(node, dict): return if 'type' not in node: return if node['type'] == "folder": # folders have children for child in node['children']: get_chrome_bookmark_urls_helper(child) if node['type'] == "url" and 'url' in node: urls.append(node['url']) # find bookmarks for node in js['roots']: get_chrome_bookmark_urls_helper(js['roots'][node]) return list(set(urls)) # unique bleachbit-4.4.2/bleachbit/CleanerML.py0000775000175000017500000003117214144024253016461 0ustar fabiofabio# vim: ts=4:sw=4:expandtab # BleachBit # Copyright (C) 2008-2021 Andrew Ziem # https://www.bleachbit.org # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . """ Create cleaners from CleanerML (markup language) """ import bleachbit from bleachbit.Action import ActionProvider from bleachbit import _ from bleachbit.General import boolstr_to_bool, getText from bleachbit.FileUtilities import expand_glob_join, listdir from bleachbit import Cleaner import logging import os import sys import xml.dom.minidom logger = logging.getLogger(__name__) def default_vars(): """Return default multi-value variables""" ret = {} if not os.name == 'nt': return ret # Expand ProgramFiles to also be ProgramW6432, etc. wowvars = (('ProgramFiles', 'ProgramW6432'), ('CommonProgramFiles', 'CommonProgramW6432')) for v1, v2 in wowvars: # Remove None, if variable is not found. # Make list unique. mylist = list({x for x in (os.getenv(v1), os.getenv(v2)) if x}) ret[v1] = mylist return ret class CleanerML: """Create a cleaner from CleanerML""" def __init__(self, pathname, xlate_cb=None): """Create cleaner from XML in pathname. If xlate_cb is set, use it as a callback for each translate-able string. """ self.action = None self.cleaner = Cleaner.Cleaner() self.option_id = None self.option_name = None self.option_description = None self.option_warning = None self.vars = default_vars() self.xlate_cb = xlate_cb if self.xlate_cb is None: self.xlate_mode = False self.xlate_cb = lambda x, y=None: None # do nothing else: self.xlate_mode = True dom = xml.dom.minidom.parse(pathname) self.handle_cleaner(dom.getElementsByTagName('cleaner')[0]) def get_cleaner(self): """Return the created cleaner""" return self.cleaner def os_match(self, os_str, platform=sys.platform): """Return boolean whether operating system matches Keyword arguments: os_str -- the required operating system as written in XML platform -- used only for unit tests """ # If blank or if in .pot-creation-mode, return true. if len(os_str) == 0 or self.xlate_mode: return True # Otherwise, check platform. # Define the current operating system. if platform == 'darwin': current_os = ('darwin', 'bsd', 'unix') elif platform.startswith('linux'): current_os = ('linux', 'unix') elif platform.startswith('openbsd'): current_os = ('bsd', 'openbsd', 'unix') elif platform.startswith('netbsd'): current_os = ('bsd', 'netbsd', 'unix') elif platform.startswith('freebsd'): current_os = ('bsd', 'freebsd', 'unix') elif platform == 'win32': current_os = ('windows') else: raise RuntimeError('Unknown operating system: %s ' % sys.platform) # Compare current OS against required OS. return os_str in current_os def handle_cleaner(self, cleaner): """ element""" if not self.os_match(cleaner.getAttribute('os')): return self.cleaner.id = cleaner.getAttribute('id') self.handle_cleaner_label(cleaner.getElementsByTagName('label')[0]) description = cleaner.getElementsByTagName('description') if description and description[0].parentNode == cleaner: self.handle_cleaner_description(description[0]) for var in cleaner.getElementsByTagName('var'): self.handle_cleaner_var(var) for option in cleaner.getElementsByTagName('option'): try: self.handle_cleaner_option(option) except: exc_msg = _( "Error in handle_cleaner_option() for cleaner id = {cleaner_id}, option XML={option_xml}") logger.exception(exc_msg.format( cleaner_id=exc_dict, option_xml=option.toxml())) self.handle_cleaner_running(cleaner.getElementsByTagName('running')) self.handle_localizations( cleaner.getElementsByTagName('localizations')) def handle_cleaner_label(self, label): """