mailman-3.2.2/0000755000175000017500000000000013445614541014261 5ustar maxkingmaxking00000000000000mailman-3.2.2/COPYING0000644000175000017500000010451413442110351015304 0ustar maxkingmaxking00000000000000 GNU GENERAL PUBLIC LICENSE Version 3, 29 June 2007 Copyright (C) 2007 Free Software Foundation, Inc. Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. Preamble The GNU General Public License is a free, copyleft license for software and other kinds of works. The licenses for most software and other practical works are designed to take away your freedom to share and change the works. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change all versions of a program--to make sure it remains free software for all its users. We, the Free Software Foundation, use the GNU General Public License for most of our software; it applies also to any other work released this way by its authors. You can apply it to your programs, too. When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for them if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs, and that you know you can do these things. To protect your rights, we need to prevent others from denying you these rights or asking you to surrender the rights. Therefore, you have certain responsibilities if you distribute copies of the software, or if you modify it: responsibilities to respect the freedom of others. For example, if you distribute copies of such a program, whether gratis or for a fee, you must pass on to the recipients the same freedoms that you received. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights. Developers that use the GNU GPL protect your rights with two steps: (1) assert copyright on the software, and (2) offer you this License giving you legal permission to copy, distribute and/or modify it. For the developers' and authors' protection, the GPL clearly explains that there is no warranty for this free software. For both users' and authors' sake, the GPL requires that modified versions be marked as changed, so that their problems will not be attributed erroneously to authors of previous versions. Some devices are designed to deny users access to install or run modified versions of the software inside them, although the manufacturer can do so. This is fundamentally incompatible with the aim of protecting users' freedom to change the software. The systematic pattern of such abuse occurs in the area of products for individuals to use, which is precisely where it is most unacceptable. Therefore, we have designed this version of the GPL to prohibit the practice for those products. If such problems arise substantially in other domains, we stand ready to extend this provision to those domains in future versions of the GPL, as needed to protect the freedom of users. Finally, every program is threatened constantly by software patents. States should not allow patents to restrict development and use of software on general-purpose computers, but in those that do, we wish to avoid the special danger that patents applied to a free program could make it effectively proprietary. To prevent this, the GPL assures that patents cannot be used to render the program non-free. The precise terms and conditions for copying, distribution and modification follow. TERMS AND CONDITIONS 0. Definitions. "This License" refers to version 3 of the GNU General Public License. "Copyright" also means copyright-like laws that apply to other kinds of works, such as semiconductor masks. "The Program" refers to any copyrightable work licensed under this License. Each licensee is addressed as "you". "Licensees" and "recipients" may be individuals or organizations. To "modify" a work means to copy from or adapt all or part of the work in a fashion requiring copyright permission, other than the making of an exact copy. The resulting work is called a "modified version" of the earlier work or a work "based on" the earlier work. A "covered work" means either the unmodified Program or a work based on the Program. To "propagate" a work means to do anything with it that, without permission, would make you directly or secondarily liable for infringement under applicable copyright law, except executing it on a computer or modifying a private copy. Propagation includes copying, distribution (with or without modification), making available to the public, and in some countries other activities as well. To "convey" a work means any kind of propagation that enables other parties to make or receive copies. Mere interaction with a user through a computer network, with no transfer of a copy, is not conveying. An interactive user interface displays "Appropriate Legal Notices" to the extent that it includes a convenient and prominently visible feature that (1) displays an appropriate copyright notice, and (2) tells the user that there is no warranty for the work (except to the extent that warranties are provided), that licensees may convey the work under this License, and how to view a copy of this License. If the interface presents a list of user commands or options, such as a menu, a prominent item in the list meets this criterion. 1. Source Code. The "source code" for a work means the preferred form of the work for making modifications to it. "Object code" means any non-source form of a work. A "Standard Interface" means an interface that either is an official standard defined by a recognized standards body, or, in the case of interfaces specified for a particular programming language, one that is widely used among developers working in that language. The "System Libraries" of an executable work include anything, other than the work as a whole, that (a) is included in the normal form of packaging a Major Component, but which is not part of that Major Component, and (b) serves only to enable use of the work with that Major Component, or to implement a Standard Interface for which an implementation is available to the public in source code form. A "Major Component", in this context, means a major essential component (kernel, window system, and so on) of the specific operating system (if any) on which the executable work runs, or a compiler used to produce the work, or an object code interpreter used to run it. The "Corresponding Source" for a work in object code form means all the source code needed to generate, install, and (for an executable work) run the object code and to modify the work, including scripts to control those activities. However, it does not include the work's System Libraries, or general-purpose tools or generally available free programs which are used unmodified in performing those activities but which are not part of the work. For example, Corresponding Source includes interface definition files associated with source files for the work, and the source code for shared libraries and dynamically linked subprograms that the work is specifically designed to require, such as by intimate data communication or control flow between those subprograms and other parts of the work. The Corresponding Source need not include anything that users can regenerate automatically from other parts of the Corresponding Source. The Corresponding Source for a work in source code form is that same work. 2. Basic Permissions. All rights granted under this License are granted for the term of copyright on the Program, and are irrevocable provided the stated conditions are met. This License explicitly affirms your unlimited permission to run the unmodified Program. The output from running a covered work is covered by this License only if the output, given its content, constitutes a covered work. This License acknowledges your rights of fair use or other equivalent, as provided by copyright law. You may make, run and propagate covered works that you do not convey, without conditions so long as your license otherwise remains in force. You may convey covered works to others for the sole purpose of having them make modifications exclusively for you, or provide you with facilities for running those works, provided that you comply with the terms of this License in conveying all material for which you do not control copyright. Those thus making or running the covered works for you must do so exclusively on your behalf, under your direction and control, on terms that prohibit them from making any copies of your copyrighted material outside their relationship with you. Conveying under any other circumstances is permitted solely under the conditions stated below. Sublicensing is not allowed; section 10 makes it unnecessary. 3. Protecting Users' Legal Rights From Anti-Circumvention Law. No covered work shall be deemed part of an effective technological measure under any applicable law fulfilling obligations under article 11 of the WIPO copyright treaty adopted on 20 December 1996, or similar laws prohibiting or restricting circumvention of such measures. When you convey a covered work, you waive any legal power to forbid circumvention of technological measures to the extent such circumvention is effected by exercising rights under this License with respect to the covered work, and you disclaim any intention to limit operation or modification of the work as a means of enforcing, against the work's users, your or third parties' legal rights to forbid circumvention of technological measures. 4. Conveying Verbatim Copies. You may convey verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice; keep intact all notices stating that this License and any non-permissive terms added in accord with section 7 apply to the code; keep intact all notices of the absence of any warranty; and give all recipients a copy of this License along with the Program. You may charge any price or no price for each copy that you convey, and you may offer support or warranty protection for a fee. 5. Conveying Modified Source Versions. You may convey a work based on the Program, or the modifications to produce it from the Program, in the form of source code under the terms of section 4, provided that you also meet all of these conditions: a) The work must carry prominent notices stating that you modified it, and giving a relevant date. b) The work must carry prominent notices stating that it is released under this License and any conditions added under section 7. This requirement modifies the requirement in section 4 to "keep intact all notices". c) You must license the entire work, as a whole, under this License to anyone who comes into possession of a copy. This License will therefore apply, along with any applicable section 7 additional terms, to the whole of the work, and all its parts, regardless of how they are packaged. This License gives no permission to license the work in any other way, but it does not invalidate such permission if you have separately received it. d) If the work has interactive user interfaces, each must display Appropriate Legal Notices; however, if the Program has interactive interfaces that do not display Appropriate Legal Notices, your work need not make them do so. A compilation of a covered work with other separate and independent works, which are not by their nature extensions of the covered work, and which are not combined with it such as to form a larger program, in or on a volume of a storage or distribution medium, is called an "aggregate" if the compilation and its resulting copyright are not used to limit the access or legal rights of the compilation's users beyond what the individual works permit. Inclusion of a covered work in an aggregate does not cause this License to apply to the other parts of the aggregate. 6. Conveying Non-Source Forms. You may convey a covered work in object code form under the terms of sections 4 and 5, provided that you also convey the machine-readable Corresponding Source under the terms of this License, in one of these ways: a) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by the Corresponding Source fixed on a durable physical medium customarily used for software interchange. b) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by a written offer, valid for at least three years and valid for as long as you offer spare parts or customer support for that product model, to give anyone who possesses the object code either (1) a copy of the Corresponding Source for all the software in the product that is covered by this License, on a durable physical medium customarily used for software interchange, for a price no more than your reasonable cost of physically performing this conveying of source, or (2) access to copy the Corresponding Source from a network server at no charge. c) Convey individual copies of the object code with a copy of the written offer to provide the Corresponding Source. This alternative is allowed only occasionally and noncommercially, and only if you received the object code with such an offer, in accord with subsection 6b. d) Convey the object code by offering access from a designated place (gratis or for a charge), and offer equivalent access to the Corresponding Source in the same way through the same place at no further charge. You need not require recipients to copy the Corresponding Source along with the object code. If the place to copy the object code is a network server, the Corresponding Source may be on a different server (operated by you or a third party) that supports equivalent copying facilities, provided you maintain clear directions next to the object code saying where to find the Corresponding Source. Regardless of what server hosts the Corresponding Source, you remain obligated to ensure that it is available for as long as needed to satisfy these requirements. e) Convey the object code using peer-to-peer transmission, provided you inform other peers where the object code and Corresponding Source of the work are being offered to the general public at no charge under subsection 6d. A separable portion of the object code, whose source code is excluded from the Corresponding Source as a System Library, need not be included in conveying the object code work. A "User Product" is either (1) a "consumer product", which means any tangible personal property which is normally used for personal, family, or household purposes, or (2) anything designed or sold for incorporation into a dwelling. In determining whether a product is a consumer product, doubtful cases shall be resolved in favor of coverage. For a particular product received by a particular user, "normally used" refers to a typical or common use of that class of product, regardless of the status of the particular user or of the way in which the particular user actually uses, or expects or is expected to use, the product. A product is a consumer product regardless of whether the product has substantial commercial, industrial or non-consumer uses, unless such uses represent the only significant mode of use of the product. "Installation Information" for a User Product means any methods, procedures, authorization keys, or other information required to install and execute modified versions of a covered work in that User Product from a modified version of its Corresponding Source. The information must suffice to ensure that the continued functioning of the modified object code is in no case prevented or interfered with solely because modification has been made. If you convey an object code work under this section in, or with, or specifically for use in, a User Product, and the conveying occurs as part of a transaction in which the right of possession and use of the User Product is transferred to the recipient in perpetuity or for a fixed term (regardless of how the transaction is characterized), the Corresponding Source conveyed under this section must be accompanied by the Installation Information. But this requirement does not apply if neither you nor any third party retains the ability to install modified object code on the User Product (for example, the work has been installed in ROM). The requirement to provide Installation Information does not include a requirement to continue to provide support service, warranty, or updates for a work that has been modified or installed by the recipient, or for the User Product in which it has been modified or installed. Access to a network may be denied when the modification itself materially and adversely affects the operation of the network or violates the rules and protocols for communication across the network. Corresponding Source conveyed, and Installation Information provided, in accord with this section must be in a format that is publicly documented (and with an implementation available to the public in source code form), and must require no special password or key for unpacking, reading or copying. 7. Additional Terms. "Additional permissions" are terms that supplement the terms of this License by making exceptions from one or more of its conditions. Additional permissions that are applicable to the entire Program shall be treated as though they were included in this License, to the extent that they are valid under applicable law. If additional permissions apply only to part of the Program, that part may be used separately under those permissions, but the entire Program remains governed by this License without regard to the additional permissions. When you convey a copy of a covered work, you may at your option remove any additional permissions from that copy, or from any part of it. (Additional permissions may be written to require their own removal in certain cases when you modify the work.) You may place additional permissions on material, added by you to a covered work, for which you have or can give appropriate copyright permission. Notwithstanding any other provision of this License, for material you add to a covered work, you may (if authorized by the copyright holders of that material) supplement the terms of this License with terms: a) Disclaiming warranty or limiting liability differently from the terms of sections 15 and 16 of this License; or b) Requiring preservation of specified reasonable legal notices or author attributions in that material or in the Appropriate Legal Notices displayed by works containing it; or c) Prohibiting misrepresentation of the origin of that material, or requiring that modified versions of such material be marked in reasonable ways as different from the original version; or d) Limiting the use for publicity purposes of names of licensors or authors of the material; or e) Declining to grant rights under trademark law for use of some trade names, trademarks, or service marks; or f) Requiring indemnification of licensors and authors of that material by anyone who conveys the material (or modified versions of it) with contractual assumptions of liability to the recipient, for any liability that these contractual assumptions directly impose on those licensors and authors. All other non-permissive additional terms are considered "further restrictions" within the meaning of section 10. If the Program as you received it, or any part of it, contains a notice stating that it is governed by this License along with a term that is a further restriction, you may remove that term. If a license document contains a further restriction but permits relicensing or conveying under this License, you may add to a covered work material governed by the terms of that license document, provided that the further restriction does not survive such relicensing or conveying. If you add terms to a covered work in accord with this section, you must place, in the relevant source files, a statement of the additional terms that apply to those files, or a notice indicating where to find the applicable terms. Additional terms, permissive or non-permissive, may be stated in the form of a separately written license, or stated as exceptions; the above requirements apply either way. 8. Termination. You may not propagate or modify a covered work except as expressly provided under this License. Any attempt otherwise to propagate or modify it is void, and will automatically terminate your rights under this License (including any patent licenses granted under the third paragraph of section 11). However, if you cease all violation of this License, then your license from a particular copyright holder is reinstated (a) provisionally, unless and until the copyright holder explicitly and finally terminates your license, and (b) permanently, if the copyright holder fails to notify you of the violation by some reasonable means prior to 60 days after the cessation. Moreover, your license from a particular copyright holder is reinstated permanently if the copyright holder notifies you of the violation by some reasonable means, this is the first time you have received notice of violation of this License (for any work) from that copyright holder, and you cure the violation prior to 30 days after your receipt of the notice. Termination of your rights under this section does not terminate the licenses of parties who have received copies or rights from you under this License. If your rights have been terminated and not permanently reinstated, you do not qualify to receive new licenses for the same material under section 10. 9. Acceptance Not Required for Having Copies. You are not required to accept this License in order to receive or run a copy of the Program. Ancillary propagation of a covered work occurring solely as a consequence of using peer-to-peer transmission to receive a copy likewise does not require acceptance. However, nothing other than this License grants you permission to propagate or modify any covered work. These actions infringe copyright if you do not accept this License. Therefore, by modifying or propagating a covered work, you indicate your acceptance of this License to do so. 10. Automatic Licensing of Downstream Recipients. Each time you convey a covered work, the recipient automatically receives a license from the original licensors, to run, modify and propagate that work, subject to this License. You are not responsible for enforcing compliance by third parties with this License. An "entity transaction" is a transaction transferring control of an organization, or substantially all assets of one, or subdividing an organization, or merging organizations. If propagation of a covered work results from an entity transaction, each party to that transaction who receives a copy of the work also receives whatever licenses to the work the party's predecessor in interest had or could give under the previous paragraph, plus a right to possession of the Corresponding Source of the work from the predecessor in interest, if the predecessor has it or can get it with reasonable efforts. You may not impose any further restrictions on the exercise of the rights granted or affirmed under this License. For example, you may not impose a license fee, royalty, or other charge for exercise of rights granted under this License, and you may not initiate litigation (including a cross-claim or counterclaim in a lawsuit) alleging that any patent claim is infringed by making, using, selling, offering for sale, or importing the Program or any portion of it. 11. Patents. A "contributor" is a copyright holder who authorizes use under this License of the Program or a work on which the Program is based. The work thus licensed is called the contributor's "contributor version". A contributor's "essential patent claims" are all patent claims owned or controlled by the contributor, whether already acquired or hereafter acquired, that would be infringed by some manner, permitted by this License, of making, using, or selling its contributor version, but do not include claims that would be infringed only as a consequence of further modification of the contributor version. For purposes of this definition, "control" includes the right to grant patent sublicenses in a manner consistent with the requirements of this License. Each contributor grants you a non-exclusive, worldwide, royalty-free patent license under the contributor's essential patent claims, to make, use, sell, offer for sale, import and otherwise run, modify and propagate the contents of its contributor version. In the following three paragraphs, a "patent license" is any express agreement or commitment, however denominated, not to enforce a patent (such as an express permission to practice a patent or covenant not to sue for patent infringement). To "grant" such a patent license to a party means to make such an agreement or commitment not to enforce a patent against the party. If you convey a covered work, knowingly relying on a patent license, and the Corresponding Source of the work is not available for anyone to copy, free of charge and under the terms of this License, through a publicly available network server or other readily accessible means, then you must either (1) cause the Corresponding Source to be so available, or (2) arrange to deprive yourself of the benefit of the patent license for this particular work, or (3) arrange, in a manner consistent with the requirements of this License, to extend the patent license to downstream recipients. "Knowingly relying" means you have actual knowledge that, but for the patent license, your conveying the covered work in a country, or your recipient's use of the covered work in a country, would infringe one or more identifiable patents in that country that you have reason to believe are valid. If, pursuant to or in connection with a single transaction or arrangement, you convey, or propagate by procuring conveyance of, a covered work, and grant a patent license to some of the parties receiving the covered work authorizing them to use, propagate, modify or convey a specific copy of the covered work, then the patent license you grant is automatically extended to all recipients of the covered work and works based on it. A patent license is "discriminatory" if it does not include within the scope of its coverage, prohibits the exercise of, or is conditioned on the non-exercise of one or more of the rights that are specifically granted under this License. You may not convey a covered work if you are a party to an arrangement with a third party that is in the business of distributing software, under which you make payment to the third party based on the extent of your activity of conveying the work, and under which the third party grants, to any of the parties who would receive the covered work from you, a discriminatory patent license (a) in connection with copies of the covered work conveyed by you (or copies made from those copies), or (b) primarily for and in connection with specific products or compilations that contain the covered work, unless you entered into that arrangement, or that patent license was granted, prior to 28 March 2007. Nothing in this License shall be construed as excluding or limiting any implied license or other defenses to infringement that may otherwise be available to you under applicable patent law. 12. No Surrender of Others' Freedom. If conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot convey a covered work so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not convey it at all. For example, if you agree to terms that obligate you to collect a royalty for further conveying from those to whom you convey the Program, the only way you could satisfy both those terms and this License would be to refrain entirely from conveying the Program. 13. Use with the GNU Affero General Public License. Notwithstanding any other provision of this License, you have permission to link or combine any covered work with a work licensed under version 3 of the GNU Affero General Public License into a single combined work, and to convey the resulting work. The terms of this License will continue to apply to the part which is the covered work, but the special requirements of the GNU Affero General Public License, section 13, concerning interaction through a network will apply to the combination as such. 14. Revised Versions of this License. The Free Software Foundation may publish revised and/or new versions of the GNU General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. Each version is given a distinguishing version number. If the Program specifies that a certain numbered version of the GNU General Public License "or any later version" applies to it, you have the option of following the terms and conditions either of that numbered version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of the GNU General Public License, you may choose any version ever published by the Free Software Foundation. If the Program specifies that a proxy can decide which future versions of the GNU General Public License can be used, that proxy's public statement of acceptance of a version permanently authorizes you to choose that version for the Program. Later license versions may give you additional or different permissions. However, no additional obligations are imposed on any author or copyright holder as a result of your choosing to follow a later version. 15. Disclaimer of Warranty. THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 16. Limitation of Liability. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. 17. Interpretation of Sections 15 and 16. If the disclaimer of warranty and limitation of liability provided above cannot be given local legal effect according to their terms, reviewing courts shall apply local law that most closely approximates an absolute waiver of all civil liability in connection with the Program, unless a warranty or assumption of liability accompanies a copy of the Program in return for a fee. END OF TERMS AND CONDITIONS How to Apply These Terms to Your New Programs If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms. To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively state the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found. Copyright (C) This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see . Also add information on how to contact you by electronic and paper mail. If the program does terminal interaction, make it output a short notice like this when it starts in an interactive mode: Copyright (C) This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. This is free software, and you are welcome to redistribute it under certain conditions; type `show c' for details. The hypothetical commands `show w' and `show c' should show the appropriate parts of the General Public License. Of course, your program's commands might be different; for a GUI interface, you would use an "about box". You should also get your employer (if you work as a programmer) or school, if any, to sign a "copyright disclaimer" for the program, if necessary. For more information on this, and how to apply and follow the GNU GPL, see . The GNU General Public License does not permit incorporating your program into proprietary programs. If your program is a subroutine library, you may consider it more useful to permit linking proprietary applications with the library. If this is what you want to do, use the GNU Lesser General Public License instead of this License. But first, please read .mailman-3.2.2/MANIFEST.in0000644000175000017500000000071713244427337016026 0ustar maxkingmaxking00000000000000include *.py include COPYING MANIFEST.in recursive-include .buildout * recursive-include contrib * recursive-include cron * recursive-include data * global-include *.txt *.rst *.po *.mo *.cfg *.sql *.zcml *.html *.ini *.eml *.dat global-exclude *.egg-info prune src/attic prune src/web prune eggs prune parts prune .tox include src/mailman/testing/config*.pck include src/mailman/database/alembic/script.py.mako include src/mailman/database/alembic/versions/*.py mailman-3.2.2/PKG-INFO0000644000175000017500000000217513445614541015363 0ustar maxkingmaxking00000000000000Metadata-Version: 1.1 Name: mailman Version: 3.2.2 Summary: Mailman -- the GNU mailing list manager Home-page: https://www.list.org Author: The Mailman Developers Author-email: mailman-developers@python.org License: GPLv3 Description: This is GNU Mailman, a mailing list management system distributed under the terms of the GNU General Public License (GPL) version 3 or later. The name of this software is spelled 'Mailman' with a leading capital 'M' but with a lower case second 'm'. Any other spelling is incorrect. Keywords: email Platform: UNKNOWN Classifier: Development Status :: 5 - Production/Stable Classifier: Intended Audience :: System Administrators Classifier: License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+) Classifier: Operating System :: POSIX Classifier: Programming Language :: Python :: 3.5 Classifier: Programming Language :: Python :: 3.6 Classifier: Programming Language :: Python :: 3.7 Classifier: Topic :: Communications :: Email :: Mailing List Servers Classifier: Topic :: Communications :: Usenet News Classifier: Topic :: Internet :: WWW/HTTP :: WSGI :: Application mailman-3.2.2/README.rst0000644000175000017500000000610613442110351015736 0ustar maxkingmaxking00000000000000================================================ Mailman - The GNU Mailing List Management System ================================================ .. image:: https://gitlab.com/mailman/mailman/badges/master/build.svg :target: https://gitlab.com/mailman/mailman/commits/master .. image:: https://readthedocs.org/projects/mailman/badge :target: https://mailman.readthedocs.io .. image:: https://img.shields.io/pypi/v/mailman.svg :target: https://pypi.org/project/mailman/ .. image:: https://img.shields.io/pypi/dm/mailman.svg :target: https://pypi.org/project/mailman/ Copyright (C) 1998-2019 by the Free Software Foundation, Inc. This is GNU Mailman, a mailing list management system distributed under the terms of the GNU General Public License (GPL) version 3 or later. The name of this software is spelled "Mailman" with a leading capital 'M' but with a lower case second 'm'. Any other spelling is incorrect. Technically speaking, you are reading the documentation for Mailman Core. The full `Mailman 3 suite `_ includes a web user interface called Postorius, a web archiver called HyperKitty, and a few other components. If you're looking for instructions on installing the full suite, read that documentation. Mailman is written in Python which is available for all platforms that Mailman is supported on, including GNU/Linux and most other Unix-like operating systems (e.g. Solaris, \*BSD, MacOSX, etc.). Mailman is not supported on Windows, although web and mail clients on any platform should be able to interact with Mailman just fine. The Mailman home page is: http://www.list.org and there is a community driven wiki at http://wiki.list.org For more information on Mailman, see the above web sites, or the :ref:`documentation provided with this software `. Table of Contents ================= .. toctree:: :glob: :maxdepth: 1 src/mailman/docs/introduction src/mailman/docs/release-notes src/mailman/docs/install src/mailman/config/docs/config src/mailman/docs/database src/mailman/docs/mta src/mailman/docs/postorius src/mailman/docs/hyperkitty src/mailman/plugins/docs/intro src/mailman/docs/contribute src/mailman/docs/STYLEGUIDE src/mailman/docs/internationalization src/mailman/docs/architecture src/mailman/docs/8-miles-high src/mailman/docs/NEWS src/mailman/docs/ACKNOWLEDGMENTS Mailman modules --------------- These documents are generated from the internal module documentation. .. toctree:: :maxdepth: 1 src/mailman/model/docs/model src/mailman/runners/docs/runners src/mailman/chains/docs/chains src/mailman/rules/docs/rules src/mailman/handlers/docs/handlers src/mailman/rest/docs/rest src/mailman/core/docs/core src/mailman/app/docs/app src/mailman/styles/docs/styles src/mailman/archiving/docs/common src/mailman/mta/docs/mta src/mailman/bin/docs/master src/mailman/commands/docs/commands contrib/README Indices and tables ================== * :ref:`genindex` * :ref:`modindex` * :ref:`search` mailman-3.2.2/conf.py0000644000175000017500000001717313421245331015560 0ustar maxkingmaxking00000000000000# -*- coding: utf-8 -*- # # GNU Mailman documentation build configuration file, created by # sphinx-quickstart on Fri Sep 23 21:30:41 2011. # # This file is execfile()d with the current directory set to its containing dir. # # Note that not all possible configuration values are present in this # autogenerated file. # # All configuration values have a default; values that are commented out # serve to show the default. import sys, os # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. #sys.path.insert(0, os.path.abspath('.')) # -- General configuration ----------------------------------------------------- # If your documentation needs a minimal Sphinx version, state it here. #needs_sphinx = '1.0' # Add any Sphinx extension module names here, as strings. They can be extensions # coming with Sphinx (named 'sphinx.ext.*') or your custom ones. extensions = ['sphinx.ext.autodoc', 'sphinx.ext.viewcode', 'sphinx.ext.graphviz'] # Add any paths that contain templates here, relative to this directory. # templates_path = ['_templates'] # The suffix of source filenames. source_suffix = '.rst' # The encoding of source files. #source_encoding = 'utf-8-sig' # The master toctree document. master_doc = 'README' # General information about the project. project = u'GNU Mailman' copyright = u'1998-2018 by the Free Software Foundation, Inc.' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the # built documents. # # The short X.Y version. import sys; sys.path.append('src') from mailman.version import VERSION version = '.'.join(VERSION.split('.')[0:2]) # The full version, including alpha/beta/rc tags. release = VERSION # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. #language = None # There are two options for replacing |today|: either, you set today to some # non-false value, then it is used: #today = '' # Else, today_fmt is used as the format for a strftime call. #today_fmt = '%B %d, %Y' # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. exclude_patterns = ['_build', 'eggs', '.tox', '.pc'] # The reST default role (used for this markup: `text`) to use for all documents. #default_role = None # If true, '()' will be appended to :func: etc. cross-reference text. #add_function_parentheses = True # If true, the current module name will be prepended to all description # unit titles (such as .. function::). #add_module_names = True # If true, sectionauthor and moduleauthor directives will be shown in the # output. They are ignored by default. #show_authors = False # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' # A list of ignored prefixes for module index sorting. #modindex_common_prefix = [] # -- Options for HTML output --------------------------------------------------- # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. html_theme = 'default' # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the # documentation. #html_theme_options = {} # Add any paths that contain custom themes here, relative to this directory. #html_theme_path = [] # The name for this set of Sphinx documents. If None, it defaults to # " v documentation". #html_title = None # A shorter title for the navigation bar. Default is the same as html_title. #html_short_title = None # The name of an image file (relative to this directory) to place at the top # of the sidebar. #html_logo = None # The name of an image file (within the static path) to use as favicon of the # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 # pixels large. #html_favicon = None # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". # html_static_path = ['_static'] # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, # using the given strftime format. #html_last_updated_fmt = '%b %d, %Y' # If true, SmartyPants will be used to convert quotes and dashes to # typographically correct entities. #html_use_smartypants = True # Custom sidebar templates, maps document names to template names. #html_sidebars = {} # Additional templates that should be rendered to pages, maps page names to # template names. #html_additional_pages = {} # If false, no module index is generated. #html_domain_indices = True # If false, no index is generated. #html_use_index = True # If true, the index is split into individual pages for each letter. #html_split_index = False # If true, links to the reST sources are added to the pages. #html_show_sourcelink = True # If true, "Created using Sphinx" is shown in the HTML footer. Default is True. #html_show_sphinx = True # If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. #html_show_copyright = True # If true, an OpenSearch description file will be output, and all pages will # contain a tag referring to it. The value of this option must be the # base URL from which the finished HTML is served. #html_use_opensearch = '' # This is the file name suffix for HTML files (e.g. ".xhtml"). #html_file_suffix = None # Output file base name for HTML help builder. htmlhelp_basename = 'GNUMailmandoc' # -- Options for LaTeX output -------------------------------------------------- # The paper size ('letter' or 'a4'). #latex_paper_size = 'letter' # The font size ('10pt', '11pt' or '12pt'). #latex_font_size = '10pt' # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, author, documentclass [howto/manual]). latex_documents = [ ('README', 'GNUMailman.tex', u'GNU Mailman Documentation', u'Barry Warsaw', 'manual'), ] # The name of an image file (relative to this directory) to place at the top of # the title page. #latex_logo = None # For "manual" documents, if this is true, then toplevel headings are parts, # not chapters. #latex_use_parts = False # If true, show page references after internal links. #latex_show_pagerefs = False # If true, show URL addresses after external links. #latex_show_urls = False # Additional stuff for the LaTeX preamble. #latex_preamble = '' # Documents to append as an appendix to all manuals. #latex_appendices = [] # If false, no module index is generated. #latex_domain_indices = True # -- Options for manual page output -------------------------------------------- # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ ('README', 'gnumailman', u'GNU Mailman Documentation', [u'Barry Warsaw'], 1) ] def index_html(): import errno cwd = os.getcwd() try: try: os.makedirs('build/sphinx/html') except OSError as error: if error.errno != errno.EEXIST: raise os.chdir('build/sphinx/html') try: os.symlink('README.html', 'index.html') print 'index.html -> README.html' except OSError as error: if error.errno != errno.EEXIST: raise finally: os.chdir(cwd) import atexit atexit.register(index_html) mailman-3.2.2/contrib/0000755000175000017500000000000013445614541015721 5ustar maxkingmaxking00000000000000mailman-3.2.2/contrib/README.rst0000644000175000017500000000135213244427337017413 0ustar maxkingmaxking00000000000000========================= Community Contributions ========================= This is a directory contain various contributions from the fantastic GNU Mailman community. We welcome and appreciate your contributions. Please be aware that the files in this directory are not officially supported by the core Mailman development team. They are also covered by their own license terms; consult those files for details. We do however require GPL compatible licenses for any contributed code appearing here. If you have any questions about these files, please contact the author of that file. The core Mailman development team may not be able to answer your questions, but we'll still try to review any issues or merge proposals related to them. mailman-3.2.2/contrib/gunicorn.py0000644000175000017500000000412313442110351020102 0ustar maxkingmaxking00000000000000# Copyright (C) 2015-2018 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Experimental Gunicorn based REST server. To use this do the following: * Install gunicorn as a Python 3 application (in a venv if necessary). * Copy this file into your Python's site-packages under the name mmgunicorn.py * Create a mailman.cfg with at least the following in it: [runner.rest] start: no * Start Mailman as normal: `mailman start` * Set the MAILMAN_CONFIG_FILE environment variable to the location of your mailman.cfg file from above. * Run: gunicorn mmgunicorn:run """ from public import public _initialized = False @public def run(environ, start_response): """Create the WSGI application. Use this if you want to integrate Mailman's REST server with an external WSGI server, such as gunicorn. Be sure to set the $MAILMAN_CONFIG_FILE environment variable. """ # Imports are here to evaluate them lazily, prevent circular imports, and # make flake8 happy. global _initialized if not _initialized: from mailman.core.initialize import initialize # First things first, initialize the system before any other imports or # other operations. It must only be initialized once though. initialize(propagate_logs=True) _initialized = True # Hook things up to WSGI. from mailman.rest.wsgiapp import make_application app = make_application() return app(environ, start_response) mailman-3.2.2/contrib/qmail-lmtp0000755000175000017500000000220613432413340017712 0ustar maxkingmaxking00000000000000#!/usr/bin/env python3 # # Written by Thomas Schneider # This script is placed in public domain. If this is not applicable, consider # it licensed under the CC-0: # try: import smtplib import sys import os lmtp = smtplib.LMTP("localhost", int(sys.argv[1])) try: # See for qmail command # docs and supplied environment variables. We need to replace "1" with an # empty string, as qmail only supports EXT, EXT2, EXT3, EXT4. arg_ext = sys.argv[2] if sys.argv[2] != "1" else "" lmtp.sendmail( os.environ['SENDER'], os.environ['EXT' + arg_ext] + "@" + os.environ['HOST'], sys.stdin.buffer.read() ) except smtplib.SMTPResponseException as e: if 400 <= e.smtp_code < 500: exit(111) # otherwise, it's either a 5xx aka permanent error or something else # is already b0rked, thus raise -> exit(100) -> have qmail return a # 5xx error else: raise except: exit(100) mailman-3.2.2/copybump.py0000755000175000017500000000534213244427337016502 0ustar maxkingmaxking00000000000000#! /usr/bin/env python3 import os import re import sys import stat import datetime FSF = 'by the Free Software Foundation, Inc.' this_year = datetime.date.today().year pyre_c = re.compile(r'# Copyright \(C\) ((?P\d{4})-)?(?P\d{4})') pyre_n = re.compile(r'# Copyright ((?P\d{4})-)?(?P\d{4})') new_c = '# Copyright (C) {}-{} {}' new_n = '# Copyright {}-{} {}' MODE = (stat.S_IRWXU | stat.S_IRWXG | stat.S_IRWXO) if '--noc' in sys.argv: pyre = pyre_n new = new_n sys.argv.remove('--noc') else: pyre = pyre_c new = new_c def do_file(path, owner): permissions = os.stat(path).st_mode & MODE with open(path) as in_file, open(path + '.out', 'w') as out_file: try: for line in in_file: mo_c = pyre_c.match(line) mo_n = pyre_n.match(line) if mo_c is None and mo_n is None: out_file.write(line) continue mo = (mo_n if mo_c is None else mo_c) start = (mo.group('end') if mo.group('start') is None else mo.group('start')) if int(start) == this_year: out_file.write(line) continue print(new.format(start, this_year, owner), file=out_file) print('=>', path) for line in in_file: out_file.write(line) except UnicodeDecodeError: print('Cannot convert path:', path) os.remove(path + '.out') return os.rename(path + '.out', path) os.chmod(path, permissions) def remove(dirs, path): try: dirs.remove(path) except ValueError: pass def do_walk(): try: owner = sys.argv[1] except IndexError: owner = FSF for root, dirs, files in os.walk('.'): if root == '.': remove(dirs, '.git') remove(dirs, '.tox') remove(dirs, 'bin') remove(dirs, 'contrib') remove(dirs, 'develop-eggs') remove(dirs, 'eggs') remove(dirs, 'parts') remove(dirs, 'gnu-COPYING-GPL') remove(dirs, '.installed.cfg') remove(dirs, '.bzrignore') remove(dirs, 'distribute_setup.py') if root == './src': remove(dirs, 'mailman.egg-info') if root == './src/mailman': remove(dirs, 'messages') for file_name in files: if os.path.splitext(file_name)[1] in ('.pyc', '.gz', '.egg'): continue path = os.path.join(root, file_name) if os.path.isfile(path): do_file(path, owner) if __name__ == '__main__': do_walk() mailman-3.2.2/coverage.ini0000644000175000017500000000066513421245331016553 0ustar maxkingmaxking00000000000000[run] branch = true parallel = true omit = setup* */showme.py .tox/*/lib/python3.*/site-packages/* .tox/*/lib/python3.*/site-packages/* */test_*.py /tmp/* /private/var/folders/* */testing/*.py [report] exclude_lines = pragma: nocover pragma: missed raise NotImplementedError raise AssertionError assert\s [paths] source = mailman .tox/*/lib/python3.*/site-packages/mailman mailman-3.2.2/cron/0000755000175000017500000000000013445614541015222 5ustar maxkingmaxking00000000000000mailman-3.2.2/cron/crontab.in.in0000755000175000017500000000176713441064212017613 0ustar maxkingmaxking00000000000000# At 8AM every day, mail reminders to admins as to pending requests. # They are less likely to ignore these reminders if they're mailed # early in the morning, but of course, this is local time... ;) 0 8 * * * @PYTHON@ -S @prefix@/cron/checkdbs # # At 9AM, send notifications to disabled members that are due to be # reminded to re-enable their accounts. 0 9 * * * @PYTHON@ -S @prefix@/cron/disabled # # Noon, mail digests for lists that do periodic as well as threshhold delivery. 0 12 * * * @PYTHON@ -S @prefix@/cron/senddigests # # Every 5 mins, try to gate news to mail. You can comment this one out # if you don't want to allow gating, or don't have any going on right now, # or want to exclusively use a callback strategy instead of polling. */5 * * * * @PYTHON@ -S @prefix@/cron/gate_news # # At 3:27am every night, regenerate the gzip'd archive file. Only # turn this on if the internal archiver is used and # GZIP_ARCHIVE_TXT_FILES is false in mm_cfg.py 27 3 * * * @PYTHON@ -S @prefix@/cron/nightly_gzip mailman-3.2.2/data/0000755000175000017500000000000013445614541015172 5ustar maxkingmaxking00000000000000mailman-3.2.2/data/PythonPowered.png0000644000175000017500000000166113244427337020515 0ustar maxkingmaxking00000000000000‰PNG  IHDRn,vå¯gAMA± üa0PLTEÿÿÿçççÞÞÞÖÖÖ½½½„„„BBB999111)))ÿÿÿc¥¤Ô¯§8tEXtSoftwareXV Version 3.10a Rev: 12/29/94 (PNG patch 1.2)Ý.IÕIDATxœ­V;–Ú0ÕÄ9˜šfè½  Ýdƴɲ…ì KÀít´tÙ3+°è|Θ+ï+ËÆ"ÇÒóÃ××’î}¶9µ×µjô ss;Ä€¸ÝÃQÝ ¸W/ͽ¿àü¡,Ë£?[80.ãšÿ¸Æ]òÕj]\ò¿äÄpBñ®ÉC~çÝçê¹1&)ç _ό̓Åx±·!¿ð­ëãœk–“Õ*}\n}³üœ¯ìd-q–†üp.ÆÁjÕ6)÷6±3_ÛFeIq"ùäòˆsW¹Y~y[nä¾Ý±‘~Rn[î»Çy1Wx‚ÄÂ…6ÒènóüŠ,}ÊÒpAƾô«ò!Ш´j‹ë dS >\Uå›ÂOøðIŠ ï¾.|“¥iáGø6Yj•*gºð{¤™ß ö1𹇠Øx¼^Ïm“u|€{ |3„µ4¶Ìïß ž¤ã«Nøxt__/xãÔ]¬>kÑãPŸè º<àPímUÿC?'z8tZFìõÿQ¬O¥8ðò;,‡ÚÛá™8pç}ШQ64x{[ãî*ÅØëšÝŠÔÍEâ§\óЯƒFeØíÏñÚI<µšÇ^5ªûÜÐy^âIªyìU£•謶Óïßìƒjæ‘P}÷ òC(U}ÆÕeõåGnŸÃ™8#hT}Äî …~ì>pÇ¡ NVê[vŸ:àüˆjšg'Rü¦5Mê»*'ͽóûñ³$—ÔsŠé_ä«B]b­_tÝ ÑQ¾ çWuu´Žfïj ƋߤGÊ/¨‡¡ê.j'"ºxÊY#;vÇ™bÌàðÕyqƒêðxPMöóôÊŠqøök]üºëÚ0ãNü¿ã £þ:ßǽrá8woÙ@2Èûþ{ótç›@[tí?üspÝÿúî¹ÆŽgý‘šÜwÂstIMEÐ.ŸäIŸIEND®B`‚mailman-3.2.2/data/gnu-head-tiny.jpg0000644000175000017500000000575113244427337020357 0ustar maxkingmaxking00000000000000ÿØÿàJFIFÿþZ CREATOR: XV Version 3.10a Rev: 12/29/94 (PNG patch 1.2) Quality = 75, Smoothing = 0 ÿÛC    $.' ",#(7),01444'9=82<.342ÿÀ V[ÿÄ ÿĵ}!1AQa"q2‘¡#B±ÁRÑð$3br‚ %&'()*456789:CDEFGHIJSTUVWXYZcdefghijstuvwxyzƒ„…†‡ˆ‰Š’“”•–—˜™š¢£¤¥¦§¨©ª²³´µ¶·¸¹ºÂÃÄÅÆÇÈÉÊÒÓÔÕÖרÙÚáâãäåæçèéêñòóôõö÷øùúÿÚ?÷ú¥¨êV¶ ‹4©çK‘ !Àyˆê3úÜÍpº¿Å+ 4Éo,2ÝgjˆTÜ#pA9‘íÇ¡85™mñe­nìF´±k¶)›kVÌ<í ä¹ã$ýî½w§x«L¾XÙ® …¤Èt ‰À;~£ á†FTdVÜGssÃ"ÉŠC2=ÅIQÉ"D…Ù°sÉà~µÎêþ:ðæ“§›§Õ-çVÞA!,FTg÷®y~(Ú΋,ócÏ™*·Ý,PŸ|ÏJétOC¬ÎPF±‡Ë@ÂMÞ`Ade¾ ÚØèw/¨üê »¸¬íd¸œ²ÆƒªX’x(’NIW­\O‘YËr|‰¥Yn.(ç•Ãp©òªÿÃsÏÍF‘àËcsqrÐ=¢Lÿ¾Ub²OÇR­9*ËÆF­È<%áû7/o¤ÚÁ1R¦xÓl¬;ƒ ùŽyÎO4‘Ç¢Á|ÓFâI¶d.ò¬-ܱ$ª7=N æ§_hÿÚ’éâþsfYBœ¬j ³}Õä‚sV/uÍ/N}—º…µ³ìóK*«Î29Æx¥M[N–ê;T¾·7)xáóF÷Qœ¹É·jÎ>Ñá®ÕÒVnŠâD à|ÇÆ s¾$ðTi·Fnï­ŽÐö2JH ¸gFæÀä†,NÞ½—A§G¢øâ;]Fû~—<E»ˆÊ!¸!n•€9䓵¹+ô©6©¤ÁvaÈÛ–XKåȬU×8ç gÚ«ë‡|ú\.Êk±#íR~â;¨ã¾õSøW—cUñ¥½Ž—<ˆ­$RÌ“<®6Æàºœã®á†%¦+Õ”az\׋<[¥ørÉžúìD”,¤nC€p¹ãÌdóŒ¼WñWQ×Ù--žÓM† ‘¯Ìe(ÁAÞX¶IÇ s×Ü×!£ë–®ýªÊfKÏ4:4Ž,:“€O9玦º—‡Q×#þÞ3µ•È™ {_Þ¤p*ù`¶HLnðF=¯‰5ß kW p%ýܯ,¶ììÈÃòAiÏ\žkéßøæËź=¼ñº­Ì‰°YO*pN O †ÀåW¬™Ò8žI,h 1n€c'>•àš›è·º®™§¥Å¤ïòÚK‡Ý5©À ŒÄÉ•ÃprsžF{ˆ.cÓˆ‚(R6¸öȤ2–•ÉÏ“Oñ¼6¶·£iŽÚà<»±€Œ¬Œy#CîëÑMxõΫ¨øSÄú\ÆÎÞæ?Þ:DD6»‰d1Ø+q´ •?ÝÆóüj·ÿ„FçP€Y>¥¦àfp$;°Ñœ0s»#“U4Ïk?tøµ/M=…›.m­mQF8Ìf*~R3´\‚3Î߀ü'àèVH´F×.b·idÒ³ùpD„“´)SÎЖíŠÔð.¢Ùhöþ$½ðþ—¤j—hfÂ&Ï"2I\n'aÚFqŒtÀ«ZÏ…®'Ñ/ÛMÙ¢þlÆÝ‘$GvÉ`›ÆÌ=Û  ‚2;qà/ ß7ö:øi'¹´·†K¹–çËi¥ `Y¶üÙû¹)ëÂÏðÖÓÀñK¯øfãS’[lJÖRN¦7AÎ0H+‘“¸Á8™¬|OÖî-VÝ´ÛA#òÛt€¯:˜‡Œ6AÍPðž•y>©y©jpÜG{ö5å1u#‘…ØW’9§kÜm 6¶É‘ÜòÌο31$’p1É'§eÑ]X¤`†{מø£Gû=œËrc»yˬ XÕŽÅ8Úä’Ÿá F+#Jø]áÏía5ŵ̑I+Ä"•Ý Šry9BHvé^²€mÎ=ê£Z¯ö‰¹US¾.Byݵ²£ðËþuå>6_ˆVÚăO½µ{ ¹%W´?fCöpE2L>mÄ¿ £®KW©Ù[ȾLï,’HÑ*HÒpX JŽsÎÏàóŸÅâ Ä*úÉÓ¬u‹&§ª˜¶ÄhVÎ qøm9'Ò¦¶ŽöÆK[”Ybš#Ê Á{W›ÚøY?á%½µ—G™˜JJÒÇå4$Sáò\'Þlåˆ {;Dû-ÔóÎé!yŒÑF©„‰ˆ9aܱÉÉú­¬ Zc9çõ¬‘ak©s›å¼[gšÛpBçÚ['ÇÜçî¯#³€É!f çrvªŽXõàzT ;9º˜:´ ‘?ê—ª}ù$ŸRG ]8“œñÍe vÛκ‚Ê'»{ge•mä‹!ð— àçžxö©æ–NÖâ K°.Û…ÈÊîSÐòÖŽÁÅdhÓêPi¶eÀU2e·¼,$çÚq»¨Á x;† —V:åõ¨M¸°¾GÂOp#dòÉùÁb2#ªœr:08¥¢¡»ó>É/“»ÍØv H8ã¸Î}x®cU°ÕÒÀÁ¦ùvоÅi$–V¸•øE.ÊA;rw…î*á³³ðý«stñOy÷W ,™r#,ÎÇ<ú}+ƒU^æHú@P EeèÀ\ç¹'W7á7½k{­XºÒŸíí1‹æ[¢ÊæŽ6€æA­‹+EºÞ]ÙǦŠ#y‚¯™þÐþœã¶zU{¿é©© ƒ&­(âÞ5$®GÞcÐ'œò,TР‘Èþ´ú(¢©êhÍ`æ$Üñ••Pq¼© ñÆ?Vêy1º˜î"ùXŒü¬=;õ¨-f–ÚH,®v—e}œ—U Ó®Ò3Óž™zXÒHŒnªÊFa‘ùW;/ˆ­<>ÖÚn±4Ë) \ yN ±*¥U·gå'§=ðãÄðMa$Úvªêpž ²‡aÁî—dÉÿtätê1YÞiÔgÔtÝïH²œ•¸„H’L벨bç9àç d’{eÈ#t§QE‡>Õ›dVÎáì62™b$’ döÀÚ[ÀÛÏa5ÄXgºFDqRÒp¸ Ÿ`{úfmq½#G9 f;JŒý 8çé?ÄqÁÉÇ5—ÝåÕž£t±Àc·)ÆF7m߸žœ¨ÂñŒs“Ó|ÒÑEQEQí#α¨c–$õ#ùú×)¬ørá!¶¸]AÞÞÒæ+¦óy¬Rep¹ÀeÆ)É ç—Óü%y¦x¦éïuë¹t«²gKh€Wf|Îür6©ã¸ü+ÖÓ w§ÑEÿÙmailman-3.2.2/data/mailman-large.jpg0000644000175000017500000001400613244427337020405 0ustar maxkingmaxking00000000000000ÿØÿàJFIFÿþXCREATOR: XV Version 3.10a Rev: 12/29/94 (PNG patch 1.2) Quality = 75, Smoothing = 0 ÿÛC    $.' ",#(7),01444'9=82<.342ÿÛC  2!!22222222222222222222222222222222222222222222222222ÿÀ]÷"ÿÄ ÿĵ}!1AQa"q2‘¡#B±ÁRÑð$3br‚ %&'()*456789:CDEFGHIJSTUVWXYZcdefghijstuvwxyzƒ„…†‡ˆ‰Š’“”•–—˜™š¢£¤¥¦§¨©ª²³´µ¶·¸¹ºÂÃÄÅÆÇÈÉÊÒÓÔÕÖרÙÚáâãäåæçèéêñòóôõö÷øùúÿÄ ÿĵw!1AQaq"2B‘¡±Á #3RðbrÑ $4á%ñ&'()*56789:CDEFGHIJSTUVWXYZcdefghijstuvwxyz‚ƒ„…†‡ˆ‰Š’“”•–—˜™š¢£¤¥¦§¨©ª²³´µ¶·¸¹ºÂÃÄÅÆÇÈÉÊÒÓÔÕÖרÙÚâãäåæçèéêòóôõö÷øùúÿÚ ?÷ú+?QÔŸN‘­žKr¤»§%OÓéš ×t»…/¡ìÎùýEO4okšû Ž*I]TBæÜ€Dñz⑮픀ׂÝqÍ;¢9eØšŠÎ¸×tëe,÷Ž vªð’™æxìí^VSŒq“ù‘ØÎ¥ÔŠêm -i+¨é÷!!A$€sX­k¬]g}ÊÂ1ÆÞÔv¢O‰¾œ QÌú!ûkâŸÝ©®'„±’z¤¬ð¼[™±Œ ±8ûûžµR]XӋϧj*‚‘)-»ø$Ræ’ݨQ›´j}êÇSEsqøºÚÞAo©$L eÊ®I#ƒŒvýEj®³§²†ûR ŒàõÕH¾¦sÂÖ†ñeú+2Oé1¾þ!žÀæ©·Œtï3™ä\ü‘à“C©Ô#„¯- þã~ŠåO‹šì:ØÛ;0Rs´ž{~§åO~µ¨HÍ=Ï“räË4½¢¦¿Rœ?ŠÔ}w:. ‰X¼Èõç¥WÎœÅ‚ÝÆJýàO­f[øJÚ) Ís4ä°o›ß¾xÿëzU¯øFôý¤˜çõ§Éãó¢óì' *Ó™¿Eþeø¯­f]ÑÜFGûØ«…sá˜dFò%hŸ+‚?þµ?E´Ô¬&k{¹|è|žàûóÎCMJW³DΕ'*sù3jŠ*–£ª[騦]Ï#ýÈe›ðôªm%vsœ¹b®Ë„€ $:“YSøNŠ_&)æ^~Hb1øâ©Í÷oúÄÆ8[UŒY Ì2ppy8ííIo{"k2Þ+{xþE8-!ÁŒžµMí¡× <»÷½4Zùõù}äë­êSè[ÀH±äzçŸÒ¤:¦¨Žš˜=ã[ü)–÷O5·œ×<…ЯžH8*ã>ÙÏòªŽúœ6ÑMi¥;\6Õ\ú?Z›¾ÿ‘|°m®D¾rÿ?оRHÏVœ3ÅsË‹$mÑ”äVwÛÜßM ˆ$X's»nqøŸZ§oaFóèÒµœŽÄ*¿(øôñ×¥R“¿s9R¦ÕíÊôó_æ¿¡¢±ôÍjIåû&£mö;àÅ|½û•ý ·|ŽzzÖÅZ’jèç©JTåË ¢Š)™ˆ@ ‚¨5w iwÛÌöqq†`¸'Œu¥¼Õ–ÂèG<.!aòȪHÏ|ñŽâ¦‹S³”·ŒñÉÇ>™õ©n/FoÖ¦”áu~¨Ï ÙˆÞXùÏý:}:ÇŸJÑ.É5)³;PÝ$prq]‚MŒU%F#¨VÌ”ô‹½9lî¼éä}¬ñüŠ ë“Ǿ e8DZن¯U¿zO§n½uD‰gáè¥1Êi#o-ƒFÌ7r{{ý?*Û³–Ṉ̃Z€<“å¸Wdc‘ÏJˆ/îÒWÔ^o‘­áÀoRăôÏøÕ‹ ^ËP¿óm.ŸpJ2DHÇ<ç ~¹ïN.ÏaWƒš~óz_O#FZÊâdŠ)‹;»¢Œ2Ë÷†HǬXƪÍ1Ã9EÄlrF3ÛÜVLv×é<6Òá¼2*H¤°3ÈÆqôüÝ-§ØmôÕX®P;JöÈÌÊÍŒà`õ$õè>yX«Ñæ²o·N—åøš§W²UŒ­‰3³±Î1žÞ⃫Øù&eŸÌ@Å ‰ðF20 žâ²¦ŽæëQÑÚÙ¯R4ó÷Ë4XaÀÆàF=†EU6·Imoò]>Úï,±#½†éÀǧ­'98ZM+½}|ß——üf}OKšÝãsG#ùJ$·|–ÆxsøÕX´} R -¼xÚÄ7–ÍŽr8>}>µZC:›©©Ü$W9v¸çn{1é×½hi©,š¥íÙ·{x$ ª®¸2œ¶:ŽÝýeùš±ƒtä×ÏÎÞDOá]2DhÝãaʳnÏâ¥5´½K1¤¨“„R 3mÃ'»Xšìlo´ùGÚÔGæfKX‹²äèzÓ”ctŒè׫V|’›¶½|‹P^i°­°‡åóØÇ"l’:ƒÇŽ*Äz…¬¼Â`±#lgçN;×?ö ‰|1{Å:Ü%ÃÉlÏ~VÇ©Oj׊ɠÐbµ¶†‘cA²à|¤ñØÏ?Öˆ·ØuiR_jîöÿƒýy—.. µPÓʱ‚p2zÕQ®i„/¡$›¡§ßévú‘íÿwœluÇøU𦔡€Ž_›®d'Þª\÷ÐΚÃrþñ»ùXÔŠòÚ`¦9ãmÝîMO\êøN fÝe+D{ϯ$œ“Ö®é1_Û‚í÷¢®Q¹öúû÷¡J[IJT¬åJwõÑ–µäÓìÚg+»îÆà3ƒÛšÂ·ì#:ΩŸQt‰>Jž{ä‘ü†*f‚][ÄÁåSö1”DÇ?Ÿ×îŽõ/tÕ¹:6Ù>1¹²A˜Ï.ò»ìk 1P{ËWéÛüÌ»/nuh¯ …¥ÂfMÒ‡QƒÕyààúvã½lÛXÁjfòÁÄ­¹”œìaÏJ}µºZÂ#AîO©îjj¸ÆÛœÕjó?uY[ˆª¨¡UBØ RÑEQ‰R÷M¶¿µ’ÞT*²KFv¶AÈ9õ§Ê54wòŨǔ;Ctäó´óÁäÔWGH@e*ÀF=ê\S6§ZPÓ§ù˜zŽš5±^8K–ŒnqÊ·××?ç5oL½‘Ù¬îÙEäC$â^9ê}—­$ºA¹¼’â{—8â^6q×܃ÓÓñªºÅµÌdÔ`ÃÜÀ@”ǘ1ÏãÛñ¨w‹æ±Ñ‘Tœ¿à?ò}|ÍÚ)±È²Æ®½fŠÔáØVUu*ÊÄf³ît=>è0xÝÅ3žµ;j©xmeY¶†ÚÇäUª¢•JVjèͰÐítÛ·žÛxÞ…J±ÏSœçúW3>±k|wÝ?–Ž Ü>÷9ïüý+¸®ÞÝ/åÔa* $lÙtã‡Ôð>õ•Ek(èz)ûG)Ö×mΗW¼x´¯6ÙiJ„l‘œ÷ùV_‡nu µkˆï.Õ"!ärx?ϨŽ*+‡“QÓôÛH IµÂ¾ô%N>^¹è9#ùSü9 ë*øÛån\*¯R3Óè={óIɹ¢Õ(ÓÃM;__]ΊêòÞÆ6æUŠ<…ÜÝ2jHfŽâ–' «„W=¯çRÕ-4¸äØß33‹qsÀãõ©|&ó®šö³¢«@çhPÁ'úçŸð­ß=º’Ã%‡Uoïvòw·äj.¥fË3 „ÛCýÐs×ò5=½Ä7P,ÐH²DÙÃ)È<â¸ý9BYëòo´Ìþ,©Èà}s[þXÓA¶Xó´ûØÏß9éÅ(MÉ•‰Â”[Of—Þ®]»¿´± n®ùÚ\ã5#\D¶ÿh.[CníƒÞ¹¿$’O§*M$3ƒ²B;¨ÉPFìʪ¦ çÂÖ6ç#Ü}žN7cúã8ùIã±â“©i4T0Jt¡Q=Þ¿ùdPÜîòŸvÆàŒ~tÓh/E™¸O´žDYùºgùW?àÙ"]6îEhÚ5“;â‚1žƒòü+:Xe5fˆ‡Kž ޹?NF(ö•2– >ÖtÛÛoS³¹º†Î6âAyÆO­)¹„[}£Ì_'ßÛ›¯8“GY#q†teaéëYj.š]æ:ª2D$ˆîÈ*y<úG犩NÎÆ4°¾Òš’ÞúúTÇs–‡8aß%sm¥Í÷‡l…µÄH˜Y°H-¼à^ÿ ðæª²oþÔBÄäåÞümÏ¿è(ç•´CxjJR‹¨•›Z§ÑEEs(†ÚI Ú§“ØÖ¶Õì•LeEà•?ˆ qŸB?eÖ¯,º5â͆x”V¹%p0=sŒôÏn(u4ÔQ·%ÊÓWýMM!Žü Êìç=ý};þ5¡UtÜ2ÙŽrÑ«ú‘“üëþök/4¢YL)ÝNÎTàúußš|Ê1LŸe*Õd–úÅÚÞÞÛø~Hnn¦ûeÅàˆM#7È ©b9Èg€jú_Iq’gÊÜy2²ÈpÅ}½‘ëÜÔª©šO(õÒïò½Î–ŠÉðËÏ'‡­^æfšS¿s³'ç=ÏQŠ’)d>!¸ŒÉ!Œ@¤&~Ps×ëV¥tŸsžT\g8_á¿àìiQXsÜOöëø¢œïB3 ^ “…'¡84°—šÖídžé›cÉ´¯Œ°<Œc¯½.rþ®Ò»}¿™·QÏšÞHÈS¹HÃt¬—ŽàYZ4W2—itiÀfùz)5kI—tDe•Ú7Æ%: }Ò‹¿?þ²Ô¯¡2¥Ëtö+ørVm7ÈpÀÞ^@?Îh£DØ—7ð©'d§9í7~üE(|(x¯ãIÿZ—¯4ûkåxòGFY‡ÃòÛçû>úX70,¥‰ c¿·ã“[^tFc˜¾`•Ï?çŠ}7±B½ZjÉéÛ¡›ckovMÍÙž##6G¿×ó®B‰Û\¿_&â?2 #¨~~Ü{Ö»*)8mäi SJI¯‰zއoq6§~ê Í–R =?pjLJÖTÖ.Õ"­Ž££ƒý+¦ª—•ªæk˜Ôc<·_Ç¥J¦£mv4–*Uy’Žêß©ÏÙZÞjÚ•åêÝ\ZØŒ²ŒaTØOrjKUºÓ|SåÊÒKñá¦ð[¶}9ãòü4F¸“I¶Ú*Ÿºà‚3ø}j´ºŽ¶ØXô͹êùéøTÚ(Ûž¬›‹ŠI«Y´­óÜË·7/c¯¬ÑO"ª/”AËp£éúTºF½=¦¼ºMÈÛ»¹vœœâ'¾;Ö¤wš°Ã½³ŽSg ç×éùКÿ–Å.í¤Œ¯Þ doÎ’ViܹÍÍ8¸&®ž²±¯¼·’è÷[°Ë¾æîr¹ç¶@ þžµ ¦—$~+1°u†)šu~›‹ÇvýMuÑMÊ)ÁðsO«öI¾kœË(CÙ¨ÛF¾öpÖ"kÏÅ:Ë<ªl-³ÔñÐqëÞ´Žƒ¨K¦l“Q}¬™kuB3r3“ÎO§ç]=#2¢–f rqB¤–á<|äïfÝûœ‘åð¼14y°Ë°§–IÆžøïœdU½oN:T71¦ÙV5W9þ9ù€8¯>§kb\¶ÞHAœZÌÿ„®%ÙÛlà–ŸáRÔR³fiINœmfßßзáèÌZº–#óŠ»@ËÓ&µ+ž_íû…`Έ6‡¾yæ› ž¾ž}ÑbGɱ€Á÷Ïùâ©JÊÉT §9NSWnû÷::Ìñ&]ïËÀ—`Úzt ã=‡ž÷šÍ˜â«×ƒ‚r;“éžžÕ{MÕ Ö¡š-’E%B:p3ÁúÓæR÷I ÒjªÕ'ЗCbÚ5¶â¥•v¶Ó4û&ÆêIdžÜ;Jª¯’pÀŽ3úÖO†.gCsaw´K„®ÜŽ? ñë]´¢®N#ž•yr»z}åC¦Y›…¡Ý"³8,Ä€X`œŽ”ƒK²YD«Wy¹V#æÆ3€jå|«±µ©üÏï)Á¥YÛJ²C+'Ýýã:ö'Í6çH²¼¸ûDÑ?›ŒnI]?à$Uê(åVµƒÛTæææwõ)>‘a"L¯n˜ î9m£ Îs‘ëSGeor$q… 9å†1ÉëSÑG*ì'VmY¶Bm hcˆÇòFQ“ÀéÖ–(!¶FòÔ"žI©jޝrmtéq½þEÏl÷üOáC²Ôpæ›P¾å]«Í{ $³:†$ú *m #“ 2…yG˜Ê'úQJ /ïUÛú¶©h–Ú“¬®dŠtYbr§q×¥SžÇ\†?ôKåv3ž=ò?_ç[ÔRpOP†&qJ/Tº=Nn9üK"âä9ãÊÏÿ¯ô¦´þ#‘È8B½‚/99 uæºj){7Ýšým^þÎ?qÍ/V¼M·¬jy!ÛÌÆ{ù~UnÛÃvPºÉ 2¸ç€ Ù¢š§É–.«VNËËA©Gˆ«ž¸§QEY˸TS[Ãr›&]}©h i´îŽz÷Âë,†[K¹­Ü€ †![8#¶*¯ög‰âp£SY#;³óéÍutVn”os®8ê©ZV~©3œ†Ï\U÷21ÜK9gƒŒwü1BèÚ„«‰&Tã– Î}±þxüº:(öh_\ŸD—ÈÈ‹ÃÖªêÒ±“±€~¢µ#Š8†#E_ ëO¢­E-Œ'Vu>'p¢Š)™…F–ðÇ)‘#Ur6’8©( i´sÚ­¬Öz¤Z¢ä»*ȼãÜõdVÝ­ÔW–É<,ô=Á÷§ËÍÆùÚÃ̽ÅχnGÚI⬜¨è:ç“÷ú‘_Y‚‡Ú[y£©¢¡·¹†ê=ð¸aÜwSèGcTµ=.RBÓý˜.6Û‚XÉž2“~Ýø­Òç4i·.W£4è¬8µK£<Τr-²\| d Ü‘žœt§ÜëFK+ Kˆ‘ÝÁF'Øç׌Tó£OªÔ½—õ­š+:;Ç—YÂë5£A’ÉÈÃwlãæ Ó¯ËWÝÒ4.쪣«1ÀIÜÊPqi>¢’$êMsSM.µ¯-²Ãþ‰ ÉvÁÈŽpsÓ§¢ûVŸQºŠÒÂxYö¹eÆî3žGÝ÷÷µµ¦Ø.h! ¹‰ÜÇÔÿ‘Y·Îì¶:ã«GšZ.Þe°¢–ŠÔá (¢€ (¢€ (¢€ (¢€ (¢€ (¢€ (¢€ (¢€ (¢€ (¢€ d°Å¦x¯FFÈά>RÞQb=²3ùPþ)Ñ-Ùž2æGå¶[²³c¹ÈïWît=6ïwh§~7m%sõÁäšd^ÒâÛ¶×;F2Ò3õÉçñ¤ÕO"ÔðVÕJþªÇ?{ãà®cÓ´«›“ûÆ!WðÉÇqÇZšÞËX×#C©ƒ{!N3žç§'Òºk{+[PñGŽê ž’§'ñ»ŽXºPVÃÓ³îõð –:e¦[h‚±ûÎyfúž¦­ÑEj’JÈᔥ7Í'vQE2OÿÙmailman-3.2.2/data/mailman.jpg0000644000175000017500000000374613244427337017326 0ustar maxkingmaxking00000000000000ÿØÿàJFIFÿþXCREATOR: XV Version 3.10a Rev: 12/29/94 (PNG patch 1.2) Quality = 75, Smoothing = 0 ÿÛC    $.' ",#(7),01444'9=82<.342ÿÛC  2!!22222222222222222222222222222222222222222222222222ÿÀ#n"ÿÄ ÿĵ}!1AQa"q2‘¡#B±ÁRÑð$3br‚ %&'()*456789:CDEFGHIJSTUVWXYZcdefghijstuvwxyzƒ„…†‡ˆ‰Š’“”•–—˜™š¢£¤¥¦§¨©ª²³´µ¶·¸¹ºÂÃÄÅÆÇÈÉÊÒÓÔÕÖרÙÚáâãäåæçèéêñòóôõö÷øùúÿÄ ÿĵw!1AQaq"2B‘¡±Á #3RðbrÑ $4á%ñ&'()*56789:CDEFGHIJSTUVWXYZcdefghijstuvwxyz‚ƒ„…†‡ˆ‰Š’“”•–—˜™š¢£¤¥¦§¨©ª²³´µ¶·¸¹ºÂÃÄÅÆÇÈÉÊÒÓÔÕÖרÙÚâãäåæçèéêòóôõö÷øùúÿÚ ?÷¾’Þm¤»Aá“ùÔG\³^›»Ú#9ÅN.¥Ve’Ù†!”äåÒ³ôǪݽ¹‰"u”y„’ÇB5›•®uÆ“”\¹.–öðäÿÛQ¸ý͵ßxÊÿ:Tº¼¸z¤PGŽ]˜Ukm}uBk eÆåûØ sÒ©¾?Ù„¡ŽSoåmgÎ;ÉÆ9â§™½V¦Ê„bùf¹^›ë¡°°ÝHãýO$ª)‹ê©x¯£~a‘ùÕX㺶Õ"”C $Çåý )`XØ lt¨m´™-¼·‹!œ>ã°É¸.G^7}~´ïä.D–²Vô^×ÌÑ:¤‘ae¶%»”`E0k±ìóz¼§%„’K}BÜÉ33C(Èä€3Û×Ô#Ky4CÆÚyYÔ2’Á†:ªýúsKš] Th5yZž…á¬]Ná-¬Xçø˜ñïRãP‘À–êIþäöªQiò½û|±DâìO¸H ªÌpOáO¼Ó®.uÁq ‘ nnFàŽ¾¸íÇ4^VÔN4”­-/ßõ- 6fÿ˜”Ç<ÿžiÑYß[ºì»2¦0Dƒš­m¤0µ–8®Œ`J‰³¶</^Ä“øUˆíµ(' ö¤š"Få`Aúçú~´Òò3”·Jiú¢ü’¬1àY’êRHåU–9ˆ,G®)c#T¿f.¯op£<·×¥Fú3Ë;‰Jº/ú¹–ÁìG·­6ÛØTáNÕ7.=ò4rGµÖM€WŽÿ â­Ì¶ö—± Fu–<$xäÁaÜví]Á½¶l¬„¡èCŒTh4¸aX×ìá%AÁÆzõ©œyµ¹®¿±Mr;?òæszE¸µñLpªœˆ·>võdŸ^¤öîiš•ÃͨB0Í{µËpy#…JºtÑ´ Œ m×fvFsÓ§çUΚ"bÒé%—9>\í“ïÜôïNÍK˜‡:r¥ìïÑvèÛïæP’âM;Zžð+óäîü¼ŒŒ€}OZŸÃû´ûA§˜¢ÜýKsŽzžO׊ÓI´‡Wó¡EiHë:îù±×œôÎ)®Ú ™ä/goÞíóÒQ³½Ê•nh88=R[vþ™‘a'Ùµ›k’†¸,&b¤.æ>§ðãÚ¥¹7šíÆ¡ bÈTĬ3ý?•iÞ_é·Ñr3€ÏN¢’8î$óŒ:lQ¤ÿëÌ~nÜþÔrôL=¶¼ò´¶¶Úÿå~ƒt…¹—D_²4p±™Ž]Iùr~œÔ­.¯ÄÉÑãædcó=)-¬õ(6ÂÚ8Á'hSÔÔË=ùo&æÐlpFä9ëV–‰;œó’s”•šoæVƒÎÃó¼d‹¤î’sõªòIª–E$'ÐZ™Ý{ò[ªZ¹wÊ%„9îH˜k¢z¤ñœ_Bmúâ²òNšòád)¯ú43‹·÷5ÚE?àD³C§é÷QŸ1Øù°5å4ÝØelT;0;È<Ÿ‘Ýä›8 ܤŽ07vsÙú+ÿLãŽöñ©ZSÎóy…õËtÞ*°}™Î{3ðá?Ø0ÎÂ(‹^IEND®B`‚mailman-3.2.2/setup.cfg0000644000175000017500000000012413445614541016077 0ustar maxkingmaxking00000000000000[upload_docs] upload_dir = build/sphinx/html [egg_info] tag_build = tag_date = 0 mailman-3.2.2/setup.py0000644000175000017500000001044213445605005015767 0ustar maxkingmaxking00000000000000# Copyright (C) 2007-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . import re import sys from setuptools import setup, find_packages from string import Template if sys.hexversion < 0x30500f0: print('Mailman requires at least Python 3.5') sys.exit(1) # Calculate the version number without importing the mailman package. with open('src/mailman/version.py') as fp: for line in fp: mo = re.match("VERSION = '(?P[^']+?)'", line) if mo: __version__ = mo.group('version') break else: print('No version number found') sys.exit(1) # Ensure that all the .mo files are generated from the corresponding .po file. # This procedure needs to be made sane, probably when the language packs are # properly split out. # Create the .mo files from the .po files. There may be errors and warnings # here and that could cause the digester.txt test to fail. ## start_dir = os.path.dirname('src/mailman/messages') ## for dirpath, dirnames, filenames in os.walk(start_dir): ## for filename in filenames: ## po_file = os.path.join(dirpath, filename) ## basename, ext = os.path.splitext(po_file) ## if ext <> '.po': ## continue ## mo_file = basename + '.mo' ## if (not os.path.exists(mo_file) or ## os.path.getmtime(po_file) > os.path.getmtime(mo_file)): ## # The mo file doesn't exist or is older than the po file. ## os.system('msgfmt -o %s %s' % (mo_file, po_file)) # XXX The 'bin/' prefix here should be configurable. template = Template('$script = mailman.bin.$script:main') scripts = set( template.substitute(script=script) for script in ('mailman', 'runner', 'master') ) setup( name = 'mailman', version = __version__, description = 'Mailman -- the GNU mailing list manager', long_description= """\ This is GNU Mailman, a mailing list management system distributed under the terms of the GNU General Public License (GPL) version 3 or later. The name of this software is spelled 'Mailman' with a leading capital 'M' but with a lower case second 'm'. Any other spelling is incorrect.""", author = 'The Mailman Developers', author_email = 'mailman-developers@python.org', license = 'GPLv3', url = 'https://www.list.org', keywords = 'email', classifiers = [ 'Development Status :: 5 - Production/Stable', 'Intended Audience :: System Administrators', 'License :: OSI Approved :: ' 'GNU General Public License v3 or later (GPLv3+)', 'Operating System :: POSIX', 'Programming Language :: Python :: 3.5', 'Programming Language :: Python :: 3.6', 'Programming Language :: Python :: 3.7', 'Topic :: Communications :: Email :: Mailing List Servers', 'Topic :: Communications :: Usenet News', 'Topic :: Internet :: WWW/HTTP :: WSGI :: Application', ], packages = find_packages('src'), package_dir = {'': 'src'}, include_package_data = True, entry_points = { 'console_scripts' : list(scripts), }, install_requires = [ 'aiosmtpd>=1.1', 'alembic', 'atpublic', 'click>=7.0', 'dnspython>=1.14.0', 'falcon>1.0.0', 'flufl.bounce', 'flufl.i18n>=2.0', 'flufl.lock>=3.1', 'importlib_resources', 'lazr.config', 'passlib', 'requests', 'sqlalchemy>=1.2.3', 'zope.component', 'zope.configuration', 'zope.event', 'zope.interface', ], ) # flake8: noqa mailman-3.2.2/src/0000755000175000017500000000000013445614541015050 5ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/0000755000175000017500000000000013445614541016466 5ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/__init__.py0000644000175000017500000000263313442110351020566 0ustar maxkingmaxking00000000000000# Copyright (C) 2009-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """The `mailman` package.""" import sys import pkgutil # This is a namespace package. __path__ = pkgutil.extend_path(__path__, __name__) # noqa: F821 # We have to initialize the i18n subsystem before anything else happens, # however, we'll initialize it differently for tests. We have to do it this # early so that module contents is set up before anything that needs it is # imported. # # Do *not* do this if we're building the documentation. if 'build_sphinx' not in sys.argv: # pragma: nocover if any('nose2' in arg for arg in sys.argv): from mailman.testing.i18n import initialize else: from mailman.core.i18n import initialize initialize() mailman-3.2.2/src/mailman/app/0000755000175000017500000000000013445614541017246 5ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/app/__init__.py0000644000175000017500000000000013244427337021347 0ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/app/bounces.py0000644000175000017500000002460713442110351021252 0ustar maxkingmaxking00000000000000# Copyright (C) 2007-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Application level bounce handling.""" import re import uuid import logging from email.mime.message import MIMEMessage from email.mime.text import MIMEText from email.utils import parseaddr from mailman.config import config from mailman.core.i18n import _ from mailman.email.message import OwnerNotification, UserNotification from mailman.interfaces.bounce import UnrecognizedBounceDisposition from mailman.interfaces.listmanager import IListManager from mailman.interfaces.pending import IPendable, IPendings from mailman.interfaces.subscriptions import ISubscriptionService from mailman.interfaces.template import ITemplateLoader from mailman.utilities.email import split_email from mailman.utilities.string import expand, oneline, wrap from public import public from string import Template from zope.component import getUtility from zope.interface import implementer log = logging.getLogger('mailman.config') elog = logging.getLogger('mailman.error') blog = logging.getLogger('mailman.bounce') DOT = '.' NL = '\n' @public def bounce_message(mlist, msg, error=None): """Bounce the message back to the original author. :param mlist: The mailing list that the message was posted to. :type mlist: `IMailingList` :param msg: The original message. :type msg: `email.message.Message` :param error: Optional exception causing the bounce. The exception instance must have a `.message` attribute. The exception *may* have a non-None `.reasons` attribute which would be a list of reasons for the rejection, and it may have a non-None `.substitutions` attribute. The latter, along with the formatted reasons will be interpolated into the message (`.reasons` gets put into the `$reasons` placeholder). :type error: RejectMessage """ # Bounce a message back to the sender, with an error message if provided # in the exception argument. .sender might be None or the empty string. if not msg.sender: # We can't bounce the message if we don't know who it's supposed to go # to. return subject = msg.get('subject', _('(no subject)')) subject = oneline(subject, mlist.preferred_language.charset) notice = (_('[No bounce details are available]') if error is None else str(error)) # Currently we always craft bounces as MIME messages. bmsg = UserNotification(msg.sender, mlist.owner_address, subject, lang=mlist.preferred_language) # BAW: Be sure you set the type before trying to attach, or you'll get # a MultipartConversionError. bmsg.set_type('multipart/mixed') txt = MIMEText(notice, _charset=mlist.preferred_language.charset) bmsg.attach(txt) bmsg.attach(MIMEMessage(msg)) bmsg.send(mlist) class _BaseVERPParser: """Base class for parsing VERP messages. Sadly not every MTA bounces VERP messages correctly, or consistently. First, the To: header is checked, then Delivered-To: (Postfix), Envelope-To: (Exim) and Apparently-To:. Note that there can be multiple headers so we need to search them all """ def __init__(self, pattern): self._pattern = pattern self._cre = re.compile(pattern, re.IGNORECASE) def get_verp(self, mlist, msg): """Extract a set of VERP bounce addresses. :param mlist: The mailing list being checked. :type mlist: `IMailingList` :param msg: The message being parsed. :type msg: `email.message.Message` :return: The set of addresses extracted from the VERP headers. :rtype: set of strings """ blocal, bdomain = split_email(mlist.bounces_address) values = set() verp_matches = set() for header in ('to', 'delivered-to', 'envelope-to', 'apparently-to'): values.update(msg.get_all(header, [])) for field in values: address = parseaddr(field)[1] if not address: # This header was empty. continue mo = self._cre.search(address) if not mo: # This did not match the VERP regexp. continue try: if blocal != mo.group('bounces'): # This was not a bounce to our mailing list. continue original_address = self._get_address(mo) except IndexError: elog.error('Bad VERP pattern: {0}'.format(self._pattern)) return set() else: if original_address is not None: verp_matches.add(original_address) return verp_matches @public class StandardVERP(_BaseVERPParser): def __init__(self): super().__init__(config.mta.verp_regexp) def _get_address(self, match_object): return '{0}@{1}'.format(*match_object.group('local', 'domain')) @public class ProbeVERP(_BaseVERPParser): def __init__(self): super().__init__(config.mta.verp_probe_regexp) def _get_address(self, match_object): # Extract the token and get the matching address. token = match_object.group('token') pendable = getUtility(IPendings).confirm(token) if pendable is None: # The token must have already been confirmed, or it may have been # evicted from the database already. return None # We had to pend the uuid as a unicode. member_id = uuid.UUID(hex=pendable['member_id']) member = getUtility(ISubscriptionService).get_member(member_id) if member is None: return None return member.address.email @implementer(IPendable) class _ProbePendable(dict): """The pendable dictionary for probe messages.""" PEND_TYPE = 'probe' @public def send_probe(member, msg): """Send a VERP probe to the member. :param member: The member to send the probe to. From this object, both the user and the mailing list can be determined. :type member: IMember :param msg: The bouncing message that caused the probe to be sent. :type msg: :return: The token representing this probe in the pendings database. :rtype: string """ mlist = getUtility(IListManager).get_by_list_id( member.mailing_list.list_id) template = getUtility(ITemplateLoader).get( 'list:user:notice:probe', mlist, language=member.preferred_language.code, # For backward compatibility. code=member.preferred_language.code, ) text = wrap(expand(template, mlist, dict( sender_email=member.subscriber.email, # For backward compatibility. address=member.address.email, email=member.address.email, owneraddr=mlist.owner_address, ))) message_id = msg['message-id'] if isinstance(message_id, bytes): message_id = message_id.decode('ascii') pendable = _ProbePendable( # We can only pend unicodes. member_id=member.member_id.hex, message_id=message_id, ) token = getUtility(IPendings).add(pendable) mailbox, domain_parts = split_email(mlist.bounces_address) probe_sender = Template(config.mta.verp_probe_format).safe_substitute( bounces=mailbox, token=token, domain=DOT.join(domain_parts), ) # Calculate the Subject header, in the member's preferred language. with _.using(member.preferred_language.code): subject = _('$mlist.display_name mailing list probe message') # Craft the probe message. This will be a multipart where the first part # is the probe text and the second part is the message that caused this # probe to be sent. probe = UserNotification(member.address.email, probe_sender, subject, lang=member.preferred_language) probe.set_type('multipart/mixed') notice = MIMEText(text, _charset=mlist.preferred_language.charset) probe.attach(notice) probe.attach(MIMEMessage(msg)) # Probes should not have the Precedence: bulk header. probe.send(mlist, envsender=probe_sender, verp=False, probe_token=token, add_precedence=False) return token @public def maybe_forward(mlist, msg): """Possibly forward bounce messages with no recognizable addresses. :param mlist: The mailing list. :type mlist: `IMailingList` :param msg: The bounce message to scan. :type msg: `Message` """ message_id = msg['message-id'] if (mlist.forward_unrecognized_bounces_to is UnrecognizedBounceDisposition.discard): blog.error('Discarding unrecognized bounce: {0}'.format(message_id)) return # The notification is either going to go to the list's administrators # (owners and moderators), or to the site administrators. Most of the # notification is exactly the same in either case. subject = _('Uncaught bounce notification') template = getUtility(ITemplateLoader).get( 'list:admin:notice:unrecognized', mlist) text = expand(template, mlist) text_part = MIMEText(text, _charset=mlist.preferred_language.charset) attachment = MIMEMessage(msg) if (mlist.forward_unrecognized_bounces_to is UnrecognizedBounceDisposition.administrators): keywords = dict(roster=mlist.administrators) elif (mlist.forward_unrecognized_bounces_to is UnrecognizedBounceDisposition.site_owner): keywords = {} else: raise AssertionError('Invalid forwarding disposition: {0}'.format( mlist.forward_unrecognized_bounces_to)) # Create the notification and send it. notice = OwnerNotification(mlist, subject, **keywords) notice.set_type('multipart/mixed') notice.attach(text_part) notice.attach(attachment) notice.send(mlist) mailman-3.2.2/src/mailman/app/commands.py0000644000175000017500000000206113442110351021403 0ustar maxkingmaxking00000000000000# Copyright (C) 2008-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Initialize the email commands.""" from mailman.config import config from mailman.interfaces.command import IEmailCommand from mailman.utilities.modules import add_components from public import public @public def initialize(): """Initialize the email commands.""" add_components('commands', IEmailCommand, config.commands) mailman-3.2.2/src/mailman/app/digests.py0000644000175000017500000001104213442110351021243 0ustar maxkingmaxking00000000000000# Copyright (C) 2015-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Digest functions.""" import os from mailman.config import config from mailman.email.message import Message from mailman.interfaces.digests import DigestFrequency from mailman.utilities.datetime import now as right_now from public import public @public def bump_digest_number_and_volume(mlist): """Bump the digest number and volume.""" now = right_now() if mlist.digest_last_sent_at is None: # There has been no previous digest. bump = False elif mlist.digest_volume_frequency == DigestFrequency.yearly: bump = (now.year > mlist.digest_last_sent_at.year) elif mlist.digest_volume_frequency == DigestFrequency.monthly: # Monthly. this_month = now.year * 100 + now.month digest_month = (mlist.digest_last_sent_at.year * 100 + mlist.digest_last_sent_at.month) bump = (this_month > digest_month) elif mlist.digest_volume_frequency == DigestFrequency.quarterly: # Quarterly. this_quarter = now.year * 100 + (now.month - 1) // 4 digest_quarter = (mlist.digest_last_sent_at.year * 100 + (mlist.digest_last_sent_at.month - 1) // 4) bump = (this_quarter > digest_quarter) elif mlist.digest_volume_frequency == DigestFrequency.weekly: this_week = now.year * 100 + now.isocalendar()[1] digest_week = (mlist.digest_last_sent_at.year * 100 + mlist.digest_last_sent_at.isocalendar()[1]) bump = (this_week > digest_week) elif mlist.digest_volume_frequency == DigestFrequency.daily: bump = (now.toordinal() > mlist.digest_last_sent_at.toordinal()) else: raise AssertionError( 'Bad DigestFrequency: {}'.format(mlist.digest_volume_frequency)) if bump: mlist.volume += 1 mlist.next_digest_number = 1 else: # Just bump the digest number. mlist.next_digest_number += 1 mlist.digest_last_sent_at = now @public def maybe_send_digest_now(mlist, *, force=False): """Send this mailing list's digest now. If there are any messages in this mailing list's digest, the digest is sent immediately, regardless of whether the size threshold has been met. When called through the subcommand `mailman send_digest` the value of .digest_send_periodic is consulted. :param mlist: The mailing list whose digest should be sent. :type mlist: IMailingList :param force: Should the digest be sent even if the size threshold hasn't been met? :type force: boolean """ mailbox_path = os.path.join(mlist.data_path, 'digest.mmdf') # Calculate the current size of the mailbox file. This will not tell # us exactly how big the resulting MIME and rfc1153 digest will # actually be, but it's the most easily available metric to decide # whether the size threshold has been reached. try: size = os.path.getsize(mailbox_path) except FileNotFoundError: size = 0 if (size >= mlist.digest_size_threshold * 1024.0 or (force and size > 0)): # Send the digest. Because we don't want to hold up this process # with crafting the digest, we're going to move the digest file to # a safe place, then craft a fake message for the DigestRunner as # a trigger for it to build and send the digest. mailbox_dest = os.path.join( mlist.data_path, 'digest.{0.volume}.{0.next_digest_number}.mmdf'.format( mlist)) volume = mlist.volume digest_number = mlist.next_digest_number bump_digest_number_and_volume(mlist) os.rename(mailbox_path, mailbox_dest) config.switchboards['digest'].enqueue( Message(), listid=mlist.list_id, digest_path=mailbox_dest, volume=volume, digest_number=digest_number) mailman-3.2.2/src/mailman/app/docs/0000755000175000017500000000000013445614541020176 5ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/app/docs/__init__.py0000644000175000017500000000000013244427337022277 0ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/app/docs/app.rst0000644000175000017500000000005413244427337021511 0ustar maxkingmaxking00000000000000=== App === .. toctree:: :glob: ./* mailman-3.2.2/src/mailman/app/docs/bans.rst0000644000175000017500000001144113244427337021656 0ustar maxkingmaxking00000000000000======================= Banning email addresses ======================= Email addresses can be banned from ever subscribing, either to a specific mailing list or globally within the Mailman system. Both explicit email addresses and email address patterns can be banned. Bans are managed through the `Ban Manager`. There are ban managers for specific lists, and there is a global ban manager. To get access to the global ban manager, adapt ``None``. >>> from mailman.interfaces.bans import IBanManager >>> global_bans = IBanManager(None) At first, no email addresses are banned globally. >>> global_bans.is_banned('anne@example.com') False To get a list-specific ban manager, adapt the mailing list object. >>> mlist = create_list('test@example.com') >>> test_bans = IBanManager(mlist) There are no bans for this particular list. >>> test_bans.is_banned('bart@example.com') False Specific bans ============= An email address can be banned from a specific mailing list by adding a ban to the list's ban manager. >>> test_bans.ban('cris@example.com') >>> test_bans.is_banned('cris@example.com') True >>> test_bans.is_banned('bart@example.com') False However, this is not a global ban. >>> global_bans.is_banned('cris@example.com') False Global bans =========== An email address can be banned globally, so that it cannot be subscribed to any mailing list. >>> global_bans.ban('dave@example.com') Because there is a global ban, Dave is also banned from the mailing list. >>> test_bans.is_banned('dave@example.com') True Even when a new mailing list is created, Dave is still banned from this list because of his global ban. >>> sample = create_list('sample@example.com') >>> sample_bans = IBanManager(sample) >>> sample_bans.is_banned('dave@example.com') True Dave is of course banned globally. >>> global_bans.is_banned('dave@example.com') True Cris however is not banned globally. >>> global_bans.is_banned('cris@example.com') False Even though Cris is not banned globally, we can add a global ban for her. >>> global_bans.ban('cris@example.com') >>> global_bans.is_banned('cris@example.com') True Cris is now banned from all mailing lists. >>> test_bans.is_banned('cris@example.com') True >>> sample_bans.is_banned('cris@example.com') True We can remove the global ban to once again just ban her address from just the test list. >>> global_bans.unban('cris@example.com') >>> global_bans.is_banned('cris@example.com') False >>> test_bans.is_banned('cris@example.com') True >>> sample_bans.is_banned('cris@example.com') False Regular expression bans ======================= Entire email address patterns can be banned, both for a specific mailing list and globally, just as specific addresses can be banned. Use this for example, when an entire domain is a spam faucet. When using a pattern, the email address must start with a caret (^). >>> test_bans.ban('^.*@example.org') Now, no one from example.org can subscribe to the test mailing list. >>> test_bans.is_banned('elle@example.org') True >>> test_bans.is_banned('eperson@example.org') True example.com addresses are not banned. >>> test_bans.is_banned('elle@example.com') False example.org addresses are not banned globally, nor for any other mailing list. >>> sample_bans.is_banned('elle@example.org') False >>> global_bans.is_banned('elle@example.org') False Of course, we can ban everyone from example.org globally too. >>> global_bans.ban('^.*@example.org') >>> sample_bans.is_banned('elle@example.org') True >>> global_bans.is_banned('elle@example.org') True We can remove the mailing list ban on the pattern, though the global ban will still be in place. >>> test_bans.unban('^.*@example.org') >>> test_bans.is_banned('elle@example.org') True >>> sample_bans.is_banned('elle@example.org') True >>> global_bans.is_banned('elle@example.org') True But once the global ban is removed, everyone from example.org can subscribe to the mailing lists. >>> global_bans.unban('^.*@example.org') >>> test_bans.is_banned('elle@example.org') False >>> sample_bans.is_banned('elle@example.org') False >>> global_bans.is_banned('elle@example.org') False Adding and removing bans ======================== It is not an error to add a ban more than once. These are just ignored. >>> test_bans.ban('fred@example.com') >>> test_bans.ban('fred@example.com') >>> test_bans.is_banned('fred@example.com') True Nor is it an error to remove a ban more than once. >>> test_bans.unban('fred@example.com') >>> test_bans.unban('fred@example.com') >>> test_bans.is_banned('fred@example.com') False mailman-3.2.2/src/mailman/app/docs/bounces.rst0000644000175000017500000001010713421245331022354 0ustar maxkingmaxking00000000000000======= Bounces ======= An important feature of Mailman is automatic bounce processing. Bounces, or message rejection ============================= Mailman can bounce messages back to the original sender. This is essentially equivalent to rejecting the message with notification. Mailing lists can bounce a message with an optional error message. >>> mlist = create_list('ant@example.com') Any message can be bounced. >>> msg = message_from_string("""\ ... To: ant@example.com ... From: aperson@example.com ... Subject: Something important ... ... I sometimes say something important. ... """) Bounce a message by passing in the original message, and an optional error message. The bounced message ends up in the virgin queue, awaiting sending to the original message author. >>> from mailman.app.bounces import bounce_message >>> bounce_message(mlist, msg) >>> from mailman.testing.helpers import get_queue_messages >>> items = get_queue_messages('virgin') >>> len(items) 1 >>> print(items[0].msg.as_string()) Subject: Something important From: ant-owner@example.com To: aperson@example.com MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="..." Message-ID: ... Date: ... Precedence: bulk --... Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit [No bounce details are available] --... Content-Type: message/rfc822 MIME-Version: 1.0 To: ant@example.com From: aperson@example.com Subject: Something important I sometimes say something important. --...-- An error message can be given when the message is bounced, and this will be included in the payload of the ``text/plain`` part. The error message must be passed in as an instance of a ``RejectMessage`` exception. >>> from mailman.interfaces.pipeline import RejectMessage >>> error = RejectMessage("This wasn't very important after all.") >>> bounce_message(mlist, msg, error) >>> items = get_queue_messages('virgin', expected_count=1) >>> print(items[0].msg.as_string()) Subject: Something important From: ant-owner@example.com To: aperson@example.com MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="..." Message-ID: ... Date: ... Precedence: bulk --... Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit This wasn't very important after all. --... Content-Type: message/rfc822 MIME-Version: 1.0 To: ant@example.com From: aperson@example.com Subject: Something important I sometimes say something important. --...-- The ``RejectMessage`` exception can also include a set of reasons, which will be interpolated into the message using the ``{reasons}`` placeholder. >>> error = RejectMessage("""This message is rejected because: ... ... $reasons ... """, [ ... 'I am not happy', ... 'You are not happy', ... 'We are not happy']) >>> bounce_message(mlist, msg, error) >>> items = get_queue_messages('virgin', expected_count=1) >>> print(items[0].msg.as_string()) Subject: Something important From: ant-owner@example.com To: aperson@example.com MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="..." Message-ID: ... Date: ... Precedence: bulk --... Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit This message is rejected because: I am not happy You are not happy We are not happy --... Content-Type: message/rfc822 MIME-Version: 1.0 To: ant@example.com From: aperson@example.com Subject: Something important I sometimes say something important. --... mailman-3.2.2/src/mailman/app/docs/lifecycle.rst0000644000175000017500000000602013244427337022667 0ustar maxkingmaxking00000000000000================================= Application level list life cycle ================================= The low-level way to create and delete a mailing list is to use the ``IListManager`` interface. This interface simply adds or removes the appropriate database entries to record the list's creation. There is a higher level interface for creating and deleting mailing lists which performs additional tasks such as: * validating the list's posting address (which also serves as the list's fully qualified name); * ensuring that the list's domain is registered; * :ref:`applying a list style ` to the new list; * creating and assigning list owners; * notifying watchers of list creation; * creating ancillary artifacts (such as the list's on-disk directory) Creating a list with owners =========================== You can also specify a list of owner email addresses. If these addresses are not yet known, they will be registered, and new users will be linked to them. :: >>> owners = [ ... 'aperson@example.com', ... 'bperson@example.com', ... 'cperson@example.com', ... 'dperson@example.com', ... ] >>> ant = create_list('ant@example.com', owners) >>> dump_list(address.email for address in ant.owners.addresses) aperson@example.com bperson@example.com cperson@example.com dperson@example.com None of the owner addresses are verified. >>> any(address.verified_on is not None ... for address in ant.owners.addresses) False However, all addresses are linked to users. >>> from mailman.interfaces.usermanager import IUserManager >>> from zope.component import getUtility >>> user_manager = getUtility(IUserManager) >>> for address in owners: ... user = user_manager.get_user(address) ... print(int(user.user_id.int), list(user.addresses)[0]) 1 aperson@example.com 2 bperson@example.com 3 cperson@example.com 4 dperson@example.com If you create a mailing list with owner addresses that are already known to the system, they won't be created again. >>> bee = create_list('bee@example.com', owners) >>> from operator import attrgetter >>> for user in sorted(bee.owners.users, key=attrgetter('user_id')): ... print(int(user.user_id.int), list(user.addresses)[0]) 1 aperson@example.com 2 bperson@example.com 3 cperson@example.com 4 dperson@example.com Deleting a list =============== Removing a mailing list deletes the list, all its subscribers, and any related artifacts. :: >>> from mailman.app.lifecycle import remove_list >>> remove_list(bee) >>> from mailman.interfaces.listmanager import IListManager >>> print(getUtility(IListManager).get('bee@example.com')) None We should now be able to completely recreate the mailing list. >>> buzz = create_list('bee@example.com', owners) >>> dump_list(address.email for address in bee.owners.addresses) aperson@example.com bperson@example.com cperson@example.com dperson@example.com mailman-3.2.2/src/mailman/app/docs/message.rst0000644000175000017500000000510013244427337022352 0ustar maxkingmaxking00000000000000======== Messages ======== Mailman has its own `Message` classes, derived from the standard ``email.message.Message`` class, but providing additional useful methods. User notifications ================== When Mailman needs to send a message to a user, it creates a ``UserNotification`` instance, and then calls the ``.send()`` method on this object. This method requires a mailing list instance. >>> mlist = create_list('test@example.com') The ``UserNotification`` constructor takes the recipient address, the sender address, an optional subject, optional body text, and optional language. >>> from mailman.email.message import UserNotification >>> msg = UserNotification( ... 'aperson@example.com', ... 'test@example.com', ... 'Something you need to know', ... 'I needed to tell you this.') >>> msg.send(mlist) The message will end up in the `virgin` queue. >>> from mailman.testing.helpers import get_queue_messages >>> messages = get_queue_messages('virgin') >>> len(messages) 1 >>> print(messages[0].msg.as_string()) MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Subject: Something you need to know From: test@example.com To: aperson@example.com Message-ID: ... Date: ... Precedence: bulk I needed to tell you this. The message above got a `Precedence: bulk` header added by default. If the message we're sending already has a `Precedence:` header, it shouldn't be changed. >>> del msg['precedence'] >>> msg['Precedence'] = 'list' >>> msg.send(mlist) Again, the message will end up in the `virgin` queue but with the original `Precedence:` header. >>> messages = get_queue_messages('virgin') >>> len(messages) 1 >>> print(messages[0].msg['precedence']) list Sometimes we want to send the message without a `Precedence:` header such as when we send a probe message. >>> del msg['precedence'] >>> msg.send(mlist, add_precedence=False) Again, the message will end up in the `virgin` queue but without the `Precedence:` header. >>> messages = get_queue_messages('virgin') >>> len(messages) 1 >>> print(messages[0].msg['precedence']) None However, if the message already has a `Precedence:` header, setting the `precedence=False` argument will have no effect. >>> msg['Precedence'] = 'junk' >>> msg.send(mlist, add_precedence=False) >>> messages = get_queue_messages('virgin') >>> len(messages) 1 >>> print(messages[0].msg['precedence']) junk mailman-3.2.2/src/mailman/app/docs/moderator.rst0000644000175000017500000003301313244427337022726 0ustar maxkingmaxking00000000000000.. _app-moderator: ============================ Application level moderation ============================ At an application level, moderation involves holding messages and membership changes for moderator approval. This utilizes the :ref:`lower level interface ` for list-centric moderation requests. Moderation is always mailing list-centric. >>> mlist = create_list('ant@example.com') >>> mlist.preferred_language = 'en' >>> mlist.display_name = 'A Test List' >>> mlist.admin_immed_notify = False We'll use the lower level API for diagnostic purposes. >>> from mailman.interfaces.requests import IListRequests >>> requests = IListRequests(mlist) Message moderation ================== Holding messages ---------------- Anne posts a message to the mailing list, but she is not a member of the list, so the message is held for moderator approval. >>> msg = message_from_string("""\ ... From: anne@example.org ... To: ant@example.com ... Subject: Something important ... Message-ID: ... ... Here's something important about our mailing list. ... """) *Holding a message* means keeping a copy of it that a moderator must approve before the message is posted to the mailing list. To hold the message, the message, its metadata, and a reason for the hold must be provided. In this case, we won't include any additional metadata. >>> from mailman.app.moderator import hold_message >>> hold_message(mlist, msg, {}, 'Needs approval') 1 We can also hold a message with some additional metadata. :: >>> msg = message_from_string("""\ ... From: bart@example.org ... To: ant@example.com ... Subject: Something important ... Message-ID: ... ... Here's something important about our mailing list. ... """) >>> msgdata = dict(sender='anne@example.com', approved=True) >>> hold_message(mlist, msg, msgdata, 'Feeling ornery') 2 Disposing of messages --------------------- The moderator can select one of several dispositions: * discard - throw the message away. * reject - bounces the message back to the original author. * defer - defer any action on the message (continue to hold it) * accept - accept the message for posting. The most trivial is to simply defer a decision for now. >>> from mailman.interfaces.action import Action >>> from mailman.app.moderator import handle_message >>> handle_message(mlist, 1, Action.defer) This leaves the message in the requests database. >>> key, data = requests.get_request(1) >>> print(key) The moderator can also discard the message. >>> handle_message(mlist, 1, Action.discard) >>> print(requests.get_request(1)) None The message can be rejected, which bounces the message back to the original sender. >>> handle_message(mlist, 2, Action.reject, 'Off topic') The message is no longer available in the requests database. >>> print(requests.get_request(2)) None And there is one message in the *virgin* queue - the rejection notice. >>> from mailman.testing.helpers import get_queue_messages >>> messages = get_queue_messages('virgin') >>> len(messages) 1 >>> print(messages[0].msg.as_string()) MIME-Version: 1.0 ... Subject: Request to mailing list "A Test List" rejected From: ant-bounces@example.com To: bart@example.org ... Your request to the ant@example.com mailing list Posting of your message titled "Something important" has been rejected by the list moderator. The moderator gave the following reason for rejecting your request: "Off topic" Any questions or comments should be directed to the list administrator at: ant-owner@example.com The bounce gets sent to the original sender. >>> for recipient in sorted(messages[0].msgdata['recipients']): ... print(recipient) bart@example.org Or the message can be approved. >>> msg = message_from_string("""\ ... From: cris@example.org ... To: ant@example.com ... Subject: Something important ... Message-ID: ... ... Here's something important about our mailing list. ... """) >>> id = hold_message(mlist, msg, {}, 'Needs approval') >>> handle_message(mlist, id, Action.accept) This places the message back into the incoming queue for further processing, however the message metadata indicates that the message has been approved. :: >>> messages = get_queue_messages('pipeline') >>> len(messages) 1 >>> print(messages[0].msg.as_string()) From: cris@example.org To: ant@example.com Subject: Something important ... >>> dump_msgdata(messages[0].msgdata) _parsemsg : False approved : True moderator_approved: True type : data version : 3 Forwarding the message ---------------------- The message can be forwarded to another address. This is helpful for getting the message into the inbox of one of the moderators. :: >>> msg = message_from_string("""\ ... From: elly@example.org ... To: ant@example.com ... Subject: Something important ... Message-ID: ... ... Here's something important about our mailing list. ... """) >>> req_id = hold_message(mlist, msg, {}, 'Needs approval') >>> handle_message(mlist, req_id, Action.discard, ... forward=['zack@example.com']) The forwarded message is in the virgin queue, destined for the moderator. :: >>> messages = get_queue_messages('virgin') >>> len(messages) 1 >>> print(messages[0].msg.as_string()) Subject: Forward of moderated message From: ant-bounces@example.com To: zack@example.com ... >>> for recipient in sorted(messages[0].msgdata['recipients']): ... print(recipient) zack@example.com Holding unsubscription requests =============================== Some lists require moderator approval for unsubscriptions. In this case, only the unsubscribing address is required. Fred is a member of the mailing list... >>> from mailman.interfaces.usermanager import IUserManager >>> from zope.component import getUtility >>> mlist.send_welcome_message = False >>> fred = getUtility(IUserManager).create_address( ... 'fred@example.com', 'Fred Person') >>> from mailman.interfaces.subscriptions import ISubscriptionManager >>> registrar = ISubscriptionManager(mlist) >>> token, token_owner, member = registrar.register( ... fred, pre_verified=True, pre_confirmed=True, pre_approved=True) >>> member on ant@example.com as MemberRole.member> ...but now that he wants to leave the mailing list, his request must be approved. >>> from mailman.app.moderator import hold_unsubscription >>> req_id = hold_unsubscription(mlist, 'fred@example.com') As with subscription requests, the unsubscription request can be deferred. >>> from mailman.app.moderator import handle_unsubscription >>> handle_unsubscription(mlist, req_id, Action.defer) >>> print(mlist.members.get_member('fred@example.com').address) Fred Person The held unsubscription can also be discarded, and the member will remain subscribed. >>> handle_unsubscription(mlist, req_id, Action.discard) >>> print(mlist.members.get_member('fred@example.com').address) Fred Person The request can be rejected, in which case a message is sent to the member, and the person remains a member of the mailing list. >>> req_id = hold_unsubscription(mlist, 'fred@example.com') >>> handle_unsubscription(mlist, req_id, Action.reject, 'No can do') >>> print(mlist.members.get_member('fred@example.com').address) Fred Person Fred gets a rejection notice. :: >>> messages = get_queue_messages('virgin') >>> len(messages) 1 >>> print(messages[0].msg.as_string()) MIME-Version: 1.0 ... Subject: Request to mailing list "A Test List" rejected From: ant-bounces@example.com To: fred@example.com ... Your request to the ant@example.com mailing list Unsubscription request has been rejected by the list moderator. The moderator gave the following reason for rejecting your request: "No can do" ... The unsubscription request can also be accepted. This removes the member from the mailing list. >>> req_id = hold_unsubscription(mlist, 'fred@example.com') >>> mlist.send_goodbye_message = False >>> handle_unsubscription(mlist, req_id, Action.accept) >>> print(mlist.members.get_member('fred@example.com')) None Notifications ============= Membership change requests -------------------------- Usually, the list administrators want to be notified when there are membership change requests they need to moderate. These notifications are sent when the list is configured to send them. >>> from mailman.interfaces.mailinglist import SubscriptionPolicy >>> mlist.admin_immed_notify = True >>> mlist.subscription_policy = SubscriptionPolicy.moderate Gwen tries to subscribe to the mailing list. >>> gwen = getUtility(IUserManager).create_address( ... 'gwen@example.com', 'Gwen Person') >>> token, token_owner, member = registrar.register( ... gwen, pre_verified=True, pre_confirmed=True) Her subscription must be approved by the list administrator, so she is not yet a member of the mailing list. >>> print(member) None >>> print(mlist.members.get_member('gwen@example.com')) None There's now a message in the virgin queue, destined for the list owner. >>> messages = get_queue_messages('virgin') >>> len(messages) 1 >>> print(messages[0].msg.as_string()) MIME-Version: 1.0 ... Subject: New subscription request to A Test List from gwen@example.com From: ant-owner@example.com To: ant-owner@example.com ... Your authorization is required for a mailing list subscription request approval: For: Gwen Person List: ant@example.com Similarly, the administrator gets notifications on unsubscription requests. Jeff is a member of the mailing list, and chooses to unsubscribe. >>> unsub_req_id = hold_unsubscription(mlist, 'jeff@example.org') >>> messages = get_queue_messages('virgin') >>> len(messages) 1 >>> print(messages[0].msg.as_string()) MIME-Version: 1.0 ... Subject: New unsubscription request from A Test List by jeff@example.org From: ant-owner@example.com To: ant-owner@example.com ... Your authorization is required for a mailing list unsubscription request approval: For: jeff@example.org List: ant@example.com Membership changes ------------------ When a new member request is accepted, the mailing list administrators can receive a membership change notice. >>> mlist.admin_notify_mchanges = True >>> mlist.admin_immed_notify = False >>> token, token_owner, member = registrar.confirm(token) >>> member on ant@example.com as MemberRole.member> >>> messages = get_queue_messages('virgin') >>> len(messages) 1 >>> print(messages[0].msg.as_string()) MIME-Version: 1.0 ... Subject: A Test List subscription notification From: noreply@example.com To: ant-owner@example.com ... Gwen Person has been successfully subscribed to A Test List. Similarly when an unsubscription request is accepted, the administrators can get a notification. >>> req_id = hold_unsubscription(mlist, 'gwen@example.com') >>> handle_unsubscription(mlist, req_id, Action.accept) >>> messages = get_queue_messages('virgin') >>> len(messages) 1 >>> print(messages[0].msg.as_string()) MIME-Version: 1.0 ... Subject: A Test List unsubscription notification From: noreply@example.com To: ant-owner@example.com ... Gwen Person has been removed from A Test List. Welcome messages ---------------- When a member is subscribed to the mailing list, they can get a welcome message. >>> mlist.admin_notify_mchanges = False >>> mlist.send_welcome_message = True >>> herb = getUtility(IUserManager).create_address( ... 'herb@example.com', 'Herb Person') >>> token, token_owner, member = registrar.register( ... herb, pre_verified=True, pre_confirmed=True, pre_approved=True) >>> messages = get_queue_messages('virgin') >>> len(messages) 1 >>> print(messages[0].msg.as_string()) MIME-Version: 1.0 ... Subject: Welcome to the "A Test List" mailing list From: ant-request@example.com To: Herb Person ... Welcome to the "A Test List" mailing list! ... Goodbye messages ---------------- Similarly, when the member's unsubscription request is approved, she'll get a goodbye message. >>> mlist.send_goodbye_message = True >>> req_id = hold_unsubscription(mlist, 'herb@example.com') >>> handle_unsubscription(mlist, req_id, Action.accept) >>> messages = get_queue_messages('virgin') >>> len(messages) 1 >>> print(messages[0].msg.as_string()) MIME-Version: 1.0 ... Subject: You have been unsubscribed from the A Test List mailing list From: ant-bounces@example.com To: herb@example.com ... mailman-3.2.2/src/mailman/app/docs/pipelines.rst0000644000175000017500000001122213421245331022705 0ustar maxkingmaxking00000000000000========= Pipelines ========= Pipelines process messages that have been accepted for posting, applying any modifications and also sending copies of the message to the archives, digests, NNTP, and outgoing queues. Pipelines are named and consist of a sequence of handlers, each of which is applied in turn. Unlike rules and chains, there is no way to stop a pipeline from processing the message once it's started. >>> mlist = create_list('test@example.com') >>> print(mlist.posting_pipeline) default-posting-pipeline >>> from mailman.core.pipelines import process For the purposes of these examples, we'll enable just one archiver. >>> from mailman.interfaces.mailinglist import IListArchiverSet >>> for archiver in IListArchiverSet(mlist).archivers: ... archiver.is_enabled = (archiver.name == 'mhonarc') Processing a message ==================== Messages hit the pipeline after they've been accepted for posting. >>> msg = message_from_string("""\ ... From: aperson@example.com ... To: test@example.com ... Subject: My first post ... Message-ID: ... X-Message-ID-Hash: 4CMWUN6BHVCMHMDAOSJZ2Q72G5M32MWB ... ... First post! ... """) >>> msgdata = {} >>> process(mlist, msg, msgdata, mlist.posting_pipeline) The message has been modified with additional headers (footer decorations come later during delivery). >>> print(msg.as_string()) From: aperson@example.com To: test@example.com Message-ID: X-Message-ID-Hash: 4CMWUN6BHVCMHMDAOSJZ2Q72G5M32MWB X-Mailman-Version: ... Precedence: list Subject: [Test] My first post List-Id: Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: First post! The message metadata has information about recipients and other stuff. However there are currently no recipients for this message. >>> dump_msgdata(msgdata) original_sender : aperson@example.com original_subject: My first post recipients : set() stripped_subject: My first post After pipeline processing, the message is now sitting in various other processing queues. :: >>> from mailman.testing.helpers import get_queue_messages >>> messages = get_queue_messages('archive') >>> len(messages) 1 >>> print(messages[0].msg.as_string()) From: aperson@example.com To: test@example.com Message-ID: X-Message-ID-Hash: 4CMWUN6BHVCMHMDAOSJZ2Q72G5M32MWB X-Mailman-Version: ... Precedence: list Subject: [Test] My first post List-Id: ... First post! >>> dump_msgdata(messages[0].msgdata) _parsemsg : False original_sender : aperson@example.com original_subject: My first post recipients : set() stripped_subject: My first post version : 3 This mailing list is not linked to an NNTP newsgroup, so there's nothing in the outgoing nntp queue. >>> messages = get_queue_messages('nntp') >>> len(messages) 0 The outgoing queue will hold the copy of the message that will actually get delivered to end recipients. :: >>> messages = get_queue_messages('out') >>> len(messages) 1 >>> print(messages[0].msg.as_string()) From: aperson@example.com To: test@example.com Message-ID: X-Message-ID-Hash: 4CMWUN6BHVCMHMDAOSJZ2Q72G5M32MWB X-Mailman-Version: ... Precedence: list Subject: [Test] My first post List-Id: ... First post! >>> dump_msgdata(messages[0].msgdata) _parsemsg : False listid : test.example.com original_sender : aperson@example.com original_subject: My first post recipients : set() stripped_subject: My first post version : 3 There's now one message in the digest mailbox, getting ready to be sent. :: >>> from mailman.testing.helpers import digest_mbox >>> digest = digest_mbox(mlist) >>> sum(1 for mboxmsg in digest) 1 >>> print(list(digest)[0].as_string()) From: aperson@example.com To: test@example.com Message-ID: X-Message-ID-Hash: 4CMWUN6BHVCMHMDAOSJZ2Q72G5M32MWB X-Mailman-Version: ... Precedence: list Subject: [Test] My first post List-Id: ... First post! mailman-3.2.2/src/mailman/app/docs/system.rst0000644000175000017500000000136013244427337022256 0ustar maxkingmaxking00000000000000=============== System versions =============== Mailman system information is available through the ``system`` object, which implements the ``ISystem`` interface. :: >>> from mailman.interfaces.system import ISystem >>> from mailman.core.system import system >>> from zope.interface.verify import verifyObject >>> verifyObject(ISystem, system) True The Mailman version is also available via the ``system`` object. >>> print(system.mailman_version) GNU Mailman ... The Python version running underneath is also available via the ``system`` object. :: # The entire python_version string is variable, so this is the best test # we can do. >>> import sys >>> system.python_version == sys.version True mailman-3.2.2/src/mailman/app/domain.py0000644000175000017500000000240513442110351021053 0ustar maxkingmaxking00000000000000# Copyright (C) 2011-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Application level domain support.""" from mailman.interfaces.domain import DomainDeletingEvent from mailman.interfaces.listmanager import IListManager from public import public from zope.component import getUtility @public def handle_DomainDeletingEvent(event): """Delete all mailing lists in a domain when the domain is deleted.""" if not isinstance(event, DomainDeletingEvent): return list_manager = getUtility(IListManager) for mailing_list in event.domain.mailing_lists: list_manager.delete(mailing_list) mailman-3.2.2/src/mailman/app/events.py0000644000175000017500000000335113442110351021111 0ustar maxkingmaxking00000000000000# Copyright (C) 2011-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Global events.""" from mailman.app import domain, membership, moderator, subscriptions from mailman.core import i18n, switchboard from mailman.languages import manager as language_manager from mailman.styles import manager as style_manager from mailman.utilities import passwords from public import public from zope import event @public def initialize(): """Initialize global event subscribers.""" event.subscribers.extend([ domain.handle_DomainDeletingEvent, i18n.handle_ConfigurationUpdatedEvent, language_manager.handle_ConfigurationUpdatedEvent, membership.handle_SubscriptionEvent, moderator.handle_ListDeletingEvent, passwords.handle_ConfigurationUpdatedEvent, style_manager.handle_ConfigurationUpdatedEvent, subscriptions.handle_ListDeletingEvent, subscriptions.handle_SubscriptionConfirmationNeededEvent, subscriptions.handle_UnsubscriptionConfirmationNeededEvent, switchboard.handle_ConfigurationUpdatedEvent, ]) mailman-3.2.2/src/mailman/app/inject.py0000644000175000017500000000711113442110351021057 0ustar maxkingmaxking00000000000000# Copyright (C) 2001-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Inject a message into a queue.""" from email import message_from_string from email.utils import formatdate, make_msgid from mailman.config import config from mailman.email.message import Message from mailman.utilities.email import add_message_hash from public import public @public def inject_message(mlist, msg, recipients=None, switchboard=None, **kws): """Inject a message into a queue. If the message does not have a Message-ID header, one is added. An Message-ID-Hash header is also always added. :param mlist: The mailing list this message is destined for. :type mlist: IMailingList :param msg: The Message object to inject. :type msg: a Message object :param recipients: Optional set of recipients to put into the message's metadata. :type recipients: sequence of strings :param switchboard: Optional name of switchboard to inject this message into. If not given, the 'in' switchboard is used. :type switchboard: string :param kws: Additional values for the message metadata. :type kws: dictionary :return: filebase of enqueued message :rtype: string """ if switchboard is None: switchboard = 'in' # Since we're crafting the message from whole cloth, let's make sure this # message has a Message-ID. if 'message-id' not in msg: msg['Message-ID'] = make_msgid() add_message_hash(msg) # Ditto for Date: as required by RFC 2822. if 'date' not in msg: msg['Date'] = formatdate(localtime=True) msg.original_size = len(msg.as_string()) msgdata = dict( listid=mlist.list_id, original_size=msg.original_size, ) msgdata.update(kws) if recipients is not None: msgdata['recipients'] = recipients return config.switchboards[switchboard].enqueue(msg, **msgdata) @public def inject_text(mlist, text, recipients=None, switchboard=None, **kws): """Turn text into a message and inject that into a queue. If the text does not have a Message-ID header, one is added. An Message-ID-Hash header is also always added. :param mlist: The mailing list this message is destined for. :type mlist: IMailingList :param text: The text of the message to inject. This will be parsed into a Message object. :type text: byte string :param recipients: Optional set of recipients to put into the message's metadata. :type recipients: sequence of strings :param switchboard: Optional name of switchboard to inject this message into. If not given, the 'in' switchboard is used. :type switchboard: string :param kws: Additional values for the message metadata. :type kws: dictionary :return: filebase of enqueued message :rtype: string """ message = message_from_string(text, Message) return inject_message(mlist, message, recipients, switchboard, **kws) mailman-3.2.2/src/mailman/app/lifecycle.py0000644000175000017500000001201113442110351021535 0ustar maxkingmaxking00000000000000# Copyright (C) 2007-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Application level list creation.""" import re import shutil import logging from contextlib import suppress from mailman.config import config from mailman.interfaces.address import IEmailValidator from mailman.interfaces.domain import ( BadDomainSpecificationError, IDomainManager) from mailman.interfaces.listmanager import IListManager from mailman.interfaces.mailinglist import InvalidListNameError from mailman.interfaces.member import MemberRole from mailman.interfaces.styles import IStyleManager from mailman.interfaces.usermanager import IUserManager from mailman.utilities.modules import call_name from public import public from zope.component import getUtility log = logging.getLogger('mailman.error') # These are the only characters allowed in list names. A more restrictive # class can be specified in config.mailman.listname_chars. _listname_chars = re.compile('[-_.+=!$*{}~0-9a-z]', re.IGNORECASE) @public def create_list(fqdn_listname, owners=None, style_name=None): """Create the named list and apply styles. The mailing may not exist yet, but the domain specified in `fqdn_listname` must exist. :param fqdn_listname: The fully qualified name for the new mailing list. :type fqdn_listname: string :param owners: The mailing list owners. :type owners: list of string email addresses :param style_name: The name of the style to apply to the newly created list. If not given, the default is taken from the configuration file. :type style_name: string :return: The new mailing list. :rtype: `IMailingList` :raises BadDomainSpecificationError: when the hostname part of `fqdn_listname` does not exist. :raises ListAlreadyExistsError: when the mailing list already exists. :raises InvalidEmailAddressError: when the fqdn email address is invalid. :raises InvalidListNameError: when the fqdn email address is valid but the listname contains disallowed characters. """ if owners is None: owners = [] # This raises InvalidEmailAddressError if the address is not a valid # posting address. Let these percolate up. getUtility(IEmailValidator).validate(fqdn_listname) listname, domain = fqdn_listname.split('@', 1) # We need to be fussier than just validating the posting address. Various # legal local-part characters will cause problems in list names. # First we check our maximally allowed set. if len(_listname_chars.sub('', listname)) > 0: raise InvalidListNameError(listname) # Then if another set is configured, check that. if config.mailman.listname_chars: try: cre = re.compile(config.mailman.listname_chars, re.IGNORECASE) except re.error as error: log.error( 'Bad config.mailman.listname_chars setting: %s: %s', config.mailman.listname_chars, getattr(error, 'msg', str(error)) ) else: if len(cre.sub('', listname)) > 0: raise InvalidListNameError(listname) if domain not in getUtility(IDomainManager): raise BadDomainSpecificationError(domain) mlist = getUtility(IListManager).create(fqdn_listname) style = getUtility(IStyleManager).get( config.styles.default if style_name is None else style_name) if style is not None: style.apply(mlist) # Coordinate with the MTA, as defined in the configuration file. call_name(config.mta.incoming).create(mlist) # Create any owners that don't yet exist, and subscribe all addresses as # owners of the mailing list. user_manager = getUtility(IUserManager) for owner_address in owners: address = user_manager.get_address(owner_address) if address is None: user = user_manager.create_user(owner_address) address = list(user.addresses)[0] mlist.subscribe(address, MemberRole.owner) return mlist @public def remove_list(mlist): """Remove the list and all associated artifacts and subscriptions.""" # Remove the list's data directory, if it exists. with suppress(FileNotFoundError): shutil.rmtree(mlist.data_path) # Delete the mailing list from the database. getUtility(IListManager).delete(mlist) # Do the MTA-specific list deletion tasks call_name(config.mta.incoming).delete(mlist) mailman-3.2.2/src/mailman/app/membership.py0000644000175000017500000001577613442110351021756 0ustar maxkingmaxking00000000000000# Copyright (C) 2007-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Application support for membership management.""" from email.utils import formataddr from mailman.app.notifications import ( send_admin_subscription_notice, send_goodbye_message, send_welcome_message) from mailman.core.i18n import _ from mailman.email.message import OwnerNotification from mailman.interfaces.address import IAddress from mailman.interfaces.bans import IBanManager from mailman.interfaces.member import ( AlreadySubscribedError, MemberRole, MembershipIsBannedError, NotAMemberError, SubscriptionEvent) from mailman.interfaces.template import ITemplateLoader from mailman.interfaces.user import IUser from mailman.interfaces.usermanager import IUserManager from mailman.utilities.string import expand from public import public from zope.component import getUtility @public def add_member(mlist, record, role=MemberRole.member): """Add a member right now. The member's subscription must be approved by whatever policy the list enforces. :param mlist: The mailing list to add the member to. :type mlist: `IMailingList` :param record: a subscription request record. :type record: RequestRecord :param role: The membership role for this subscription. :type role: `MemberRole` :return: The just created member. :rtype: `IMember` :raises AlreadySubscribedError: if the user is already subscribed to the mailing list. :raises InvalidEmailAddressError: if the email address is not valid. :raises MembershipIsBannedError: if the membership is not allowed. """ # Check to see if the email address is banned. if IBanManager(mlist).is_banned(record.email): raise MembershipIsBannedError(mlist, record.email) # Make sure there is a user linked with the given address. user_manager = getUtility(IUserManager) user = user_manager.make_user(record.email, record.display_name) user.preferences.preferred_language = record.language # Subscribe the address, not the user. # We're looking for two versions of the email address, the case # preserved version and the case insensitive version. We'll # subscribe the version with matching case if it exists, otherwise # we'll use one of the matching case-insensitively ones. It's # undefined which one we pick. case_preserved = None case_insensitive = None for address in user.addresses: if address.original_email == record.email: case_preserved = address if address.email == record.email.lower(): # pragma: no branch case_insensitive = address assert case_preserved is not None or case_insensitive is not None, ( 'Could not find a linked address for: {}'.format(record.email)) address = (case_preserved if case_preserved is not None else case_insensitive) # Create the member and set the appropriate preferences. It's # possible we're subscribing the lower cased version of the address; # if that's already subscribed re-issue the exception with the correct # email address (i.e. the one passed in here). try: member = mlist.subscribe(address, role) except AlreadySubscribedError as error: raise AlreadySubscribedError( error.fqdn_listname, record.email, error.role) member.preferences.preferred_language = record.language member.preferences.delivery_mode = record.delivery_mode # Check for and remove nonmember subscriptions of the user to this list. if role is MemberRole.member: for address in user.addresses: nonmember = mlist.nonmembers.get_member(address.email) if nonmember is not None: nonmember.unsubscribe() return member @public def delete_member(mlist, email, admin_notif=None, userack=None): """Delete a member right now. :param mlist: The mailing list to remove the member from. :type mlist: `IMailingList` :param email: The email address to unsubscribe. :type email: string :param admin_notif: Whether the list administrator should be notified that this member was deleted. :type admin_notif: bool, or None to let the mailing list's `admin_notify_mchange` attribute decide. :raises NotAMemberError: if the address is not a member of the mailing list. """ if userack is None: userack = mlist.send_goodbye_message if admin_notif is None: admin_notif = mlist.admin_notify_mchanges # Delete a member, for which we know the approval has been made. member = mlist.members.get_member(email) if member is None: raise NotAMemberError(mlist, email) language = member.preferred_language member.unsubscribe() # And send an acknowledgement to the user... if userack: send_goodbye_message(mlist, email, language) # ...and to the administrator. if admin_notif: user = getUtility(IUserManager).get_user(email) display_name = user.display_name subject = _('$mlist.display_name unsubscription notification') text = expand(getUtility(ITemplateLoader).get( 'list:admin:notice:unsubscribe', mlist), mlist, dict( member=formataddr((display_name, email)), )) msg = OwnerNotification(mlist, subject, text, roster=mlist.administrators) msg.send(mlist) @public def handle_SubscriptionEvent(event): if not isinstance(event, SubscriptionEvent): return member = event.member # Only send notifications if a member (as opposed to a moderator, # non-member, or owner) is being subscribed. if member.role is not MemberRole.member: return mlist = member.mailing_list # Maybe send the list administrators a notification. if mlist.admin_notify_mchanges: subscriber = member.subscriber if IAddress.providedBy(subscriber): address = subscriber.email display_name = subscriber.display_name else: assert IUser.providedBy(subscriber) address = subscriber.preferred_address.email display_name = subscriber.display_name send_admin_subscription_notice(mlist, address, display_name) # Maybe send a welcome message to the new member. if mlist.send_welcome_message: send_welcome_message(mlist, member, member.preferred_language) mailman-3.2.2/src/mailman/app/moderator.py0000644000175000017500000002574513442110351021614 0ustar maxkingmaxking00000000000000# Copyright (C) 2007-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Application support for moderators.""" import time import logging from email.utils import formatdate, getaddresses, make_msgid from mailman.app.membership import delete_member from mailman.config import config from mailman.core.i18n import _ from mailman.email.message import UserNotification from mailman.interfaces.action import Action from mailman.interfaces.listmanager import ListDeletingEvent from mailman.interfaces.member import NotAMemberError from mailman.interfaces.messages import IMessageStore from mailman.interfaces.requests import IListRequests, RequestType from mailman.interfaces.template import ITemplateLoader from mailman.utilities.datetime import now from mailman.utilities.string import expand, wrap from public import public from zope.component import getUtility NL = '\n' vlog = logging.getLogger('mailman.vette') slog = logging.getLogger('mailman.subscribe') @public def hold_message(mlist, msg, msgdata=None, reason=None): """Hold a message for moderator approval. The message is added to the mailing list's request database. :param mlist: The mailing list to hold the message on. :param msg: The message to hold. :param msgdata: Optional message metadata to hold. If not given, a new metadata dictionary is created and held with the message. :param reason: Optional string reason why the message is being held. If not given, the empty string is used. :return: An id used to handle the held message later. """ if msgdata is None: msgdata = {} else: # Make a copy of msgdata so that subsequent changes won't corrupt the # request database. TBD: remove the `filebase' key since this will # not be relevant when the message is resurrected. msgdata = msgdata.copy() if reason is None: reason = '' # Add the message to the message store. It is required to have a # Message-ID header. message_id = msg.get('message-id') if message_id is None: msg['Message-ID'] = message_id = make_msgid() elif isinstance(message_id, bytes): message_id = message_id.decode('ascii') getUtility(IMessageStore).add(msg) # Prepare the message metadata with some extra information needed only by # the moderation interface. msgdata['_mod_message_id'] = message_id msgdata['_mod_listid'] = mlist.list_id msgdata['_mod_sender'] = msg.sender # The subject can sometimes be a Header instance. Stringify it. msgdata['_mod_subject'] = str(msg.get('subject', _('(no subject)'))) msgdata['_mod_reason'] = reason msgdata['_mod_hold_date'] = now().isoformat() # Now hold this request. We'll use the message_id as the key. requestsdb = IListRequests(mlist) request_id = requestsdb.hold_request( RequestType.held_message, message_id, msgdata) return request_id @public def handle_message(mlist, id, action, comment=None, forward=None): message_store = getUtility(IMessageStore) requestdb = IListRequests(mlist) key, msgdata = requestdb.get_request(id) # Handle the action. rejection = None message_id = msgdata['_mod_message_id'] sender = msgdata['_mod_sender'] subject = msgdata['_mod_subject'] keep = False if action in (Action.defer, Action.hold): # Nothing to do, but preserve the message for later. keep = True elif action is Action.discard: rejection = 'Discarded' elif action is Action.reject: rejection = 'Refused' member = mlist.members.get_member(sender) if member: language = member.preferred_language else: language = None send_rejection( mlist, _('Posting of your message titled "$subject"'), sender, comment or _('[No reason given]'), language) elif action is Action.accept: # Start by getting the message from the message store. msg = message_store.get_message_by_id(message_id) # Delete moderation-specific entries from the message metadata. for key in list(msgdata): if key.startswith('_mod_'): del msgdata[key] # Add some metadata to indicate this message has now been approved. msgdata['approved'] = True msgdata['moderator_approved'] = True # Calculate a new filebase for the approved message, otherwise # delivery errors will cause duplicates. if 'filebase' in msgdata: del msgdata['filebase'] # Queue the file for delivery. Trying to deliver the message directly # here can lead to a huge delay in web turnaround. Log the moderation # and add a header. msg['X-Mailman-Approved-At'] = formatdate( time.mktime(now().timetuple()), localtime=True) vlog.info('held message approved, message-id: %s', msg.get('message-id', 'n/a')) # Stick the message back in the incoming queue for further # processing. config.switchboards['pipeline'].enqueue(msg, _metadata=msgdata) else: raise AssertionError('Unexpected action: {0}'.format(action)) # Forward the message. if forward: # Get a copy of the original message from the message store. msg = message_store.get_message_by_id(message_id) # It's possible the forwarding address list is a comma separated list # of display_name/address pairs. addresses = [addr[1] for addr in getaddresses(forward)] language = mlist.preferred_language if len(addresses) == 1: # If the address getting the forwarded message is a member of # the list, we want the headers of the outer message to be # encoded in their language. Otherwise it'll be the preferred # language of the mailing list. This is better than sending a # separate message per recipient. member = mlist.members.get_member(addresses[0]) if member: language = member.preferred_language with _.using(language.code): fmsg = UserNotification( addresses, mlist.bounces_address, _('Forward of moderated message'), lang=language) fmsg.set_type('message/rfc822') fmsg.attach(msg) fmsg.send(mlist) # Delete the request if it's not being kept. if not keep: requestdb.delete_request(id) # Log the rejection if rejection: note = """%s: %s posting: \tFrom: %s \tSubject: %s""" if comment: note += '\n\tReason: ' + comment vlog.info(note, mlist.fqdn_listname, rejection, sender, subject) @public def hold_unsubscription(mlist, email): data = dict(email=email) requestsdb = IListRequests(mlist) request_id = requestsdb.hold_request( RequestType.unsubscription, email, data) vlog.info('%s: held unsubscription request from %s', mlist.fqdn_listname, email) # Possibly notify the administrator of the hold if mlist.admin_immed_notify: subject = _( 'New unsubscription request from $mlist.display_name by $email') template = getUtility(ITemplateLoader).get( 'list:admin:action:unsubscribe', mlist) text = wrap(expand(template, mlist, dict( # For backward compatibility. mailing_list=mlist, member=email, email=email, ))) # This message should appear to come from the -owner so as # to avoid any useless bounce processing. msg = UserNotification( mlist.owner_address, mlist.owner_address, subject, text, mlist.preferred_language) msg.send(mlist) return request_id @public def handle_unsubscription(mlist, id, action, comment=None): requestdb = IListRequests(mlist) key, data = requestdb.get_request(id) email = data['email'] if action is Action.defer: # Nothing to do. return elif action is Action.discard: # Nothing to do except delete the request from the database. pass elif action is Action.reject: key, data = requestdb.get_request(id) send_rejection( mlist, _('Unsubscription request'), email, comment or _('[No reason given]')) elif action is Action.accept: key, data = requestdb.get_request(id) try: delete_member(mlist, email) except NotAMemberError: # User has already been unsubscribed. pass slog.info('%s: deleted %s', mlist.fqdn_listname, email) else: raise AssertionError('Unexpected action: {}'.format(action)) # Delete the request from the database. requestdb.delete_request(id) @public def send_rejection(mlist, request, recip, comment, origmsg=None, lang=None): # As this message is going to the requester, try to set the language to # his/her language choice, if they are a member. Otherwise use the list's # preferred language. display_name = mlist.display_name # noqa: F841 if lang is None: member = mlist.members.get_member(recip) lang = (mlist.preferred_language if member is None else member.preferred_language) template = getUtility(ITemplateLoader).get( 'list:user:notice:refuse', mlist) text = wrap(expand(template, mlist, dict( language=lang.code, reason=comment, # For backward compatibility. request=request, adminaddr=mlist.owner_address, ))) with _.using(lang.code): # add in original message, but not wrap/filled if origmsg: text = NL.join( [text, '---------- ' + _('Original Message') + ' ----------', str(origmsg) ]) subject = _('Request to mailing list "$display_name" rejected') msg = UserNotification(recip, mlist.bounces_address, subject, text, lang) msg.send(mlist) @public def handle_ListDeletingEvent(event): if not isinstance(event, ListDeletingEvent): return # Get the held requests database for the mailing list. Since the mailing # list is about to get deleted, we can delete all associated requests. requestsdb = IListRequests(event.mailing_list) for request in requestsdb.held_requests: requestsdb.delete_request(request.id) mailman-3.2.2/src/mailman/app/notifications.py0000644000175000017500000001046213442110351022457 0ustar maxkingmaxking00000000000000# Copyright (C) 2007-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Sending notifications.""" import logging from email.utils import formataddr from lazr.config import as_boolean from mailman.config import config from mailman.core.i18n import _ from mailman.email.message import OwnerNotification, UserNotification from mailman.interfaces.member import DeliveryMode from mailman.interfaces.template import ITemplateLoader from mailman.utilities.string import expand, wrap from public import public from zope.component import getUtility log = logging.getLogger('mailman.error') @public def send_welcome_message(mlist, member, language, text=''): """Send a welcome message to a subscriber. Prepending to the standard welcome message template is the mailing list's welcome message, if there is one. :param mlist: The mailing list. :type mlist: IMailingList :param member: The member to send the welcome message to. :param address: IMember :param language: The language of the response. :type language: ILanguage """ welcome_message = wrap(getUtility(ITemplateLoader).get( 'list:user:notice:welcome', mlist, language=language.code)) display_name = member.display_name # Get the text from the template. text = expand(welcome_message, mlist, dict( user_name=display_name, user_email=member.address.email, # For backward compatibility. user_address=member.address.email, fqdn_listname=mlist.fqdn_listname, list_name=mlist.display_name, list_requests=mlist.request_address, )) digmode = ('' # noqa: F841 if member.delivery_mode is DeliveryMode.regular else _(' (Digest mode)')) msg = UserNotification( formataddr((display_name, member.address.email)), mlist.request_address, _('Welcome to the "$mlist.display_name" mailing list${digmode}'), text, language) msg['X-No-Archive'] = 'yes' msg.send(mlist, verp=as_boolean(config.mta.verp_personalized_deliveries)) @public def send_goodbye_message(mlist, address, language): """Send a goodbye message to a subscriber. Prepending to the standard goodbye message template is the mailing list's goodbye message, if there is one. :param mlist: the mailing list :type mlist: IMailingList :param address: The address to respond to :type address: string :param language: the language of the response :type language: string """ goodbye_message = wrap(getUtility(ITemplateLoader).get( 'list:user:notice:goodbye', mlist, language=language.code)) msg = UserNotification( address, mlist.bounces_address, _('You have been unsubscribed from the $mlist.display_name ' 'mailing list'), goodbye_message, language) msg.send(mlist, verp=as_boolean(config.mta.verp_personalized_deliveries)) @public def send_admin_subscription_notice(mlist, address, display_name): """Send the list administrators a subscription notice. :param mlist: The mailing list. :type mlist: IMailingList :param address: The address being subscribed. :type address: string :param display_name: The name of the subscriber. :type display_name: string """ with _.using(mlist.preferred_language.code): subject = _('$mlist.display_name subscription notification') text = expand( getUtility(ITemplateLoader).get('list:admin:notice:subscribe', mlist), mlist, dict( member=formataddr((display_name, address)), )) msg = OwnerNotification(mlist, subject, text, roster=mlist.administrators) msg.send(mlist) mailman-3.2.2/src/mailman/app/replybot.py0000644000175000017500000000406213442110351021445 0ustar maxkingmaxking00000000000000# Copyright (C) 2007-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Application level auto-reply code.""" from public import public @public def can_acknowledge(msg): """A boolean specifying whether this message can be acknowledged. There are several reasons why a message should not be acknowledged, mostly related to competing standards or common practices. These include: * The message has a X-No-Ack header with any value * The message has an X-Ack header with a 'no' value * The message has a Precedence header * The message has an Auto-Submitted header and that header does not have a value of 'no' * The message has an empty Return-Path header, e.g. <> * The message has any RFC 2369 headers (i.e. List-* headers) :param msg: a Message object. :return: Boolean specifying whether the message can be acknowledged or not (which is different from whether it will be acknowledged). """ # I wrote it this way for clarity and consistency with the docstring. for header in msg.keys(): if header in ('x-no-ack', 'precedence'): return False if header.lower().startswith('list-'): return False if msg.get('x-ack', '').lower() == 'no': return False if msg.get('auto-submitted', 'no').lower() != 'no': return False if msg.get('return-path') == '<>': return False return True mailman-3.2.2/src/mailman/app/subscriptions.py0000644000175000017500000005723613442110351022527 0ustar maxkingmaxking00000000000000# Copyright (C) 2009-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Handle subscriptions.""" import uuid import logging from datetime import timedelta from email.utils import formataddr from enum import Enum from mailman.app.membership import delete_member from mailman.app.workflow import Workflow from mailman.core.i18n import _ from mailman.database.transaction import flush from mailman.email.message import UserNotification from mailman.interfaces.address import IAddress from mailman.interfaces.bans import IBanManager from mailman.interfaces.listmanager import ListDeletingEvent from mailman.interfaces.mailinglist import SubscriptionPolicy from mailman.interfaces.member import ( AlreadySubscribedError, MemberRole, MembershipIsBannedError, NotAMemberError) from mailman.interfaces.pending import IPendable, IPendings from mailman.interfaces.subscriptions import ( ISubscriptionManager, ISubscriptionService, SubscriptionConfirmationNeededEvent, SubscriptionPendingError, TokenOwner, UnsubscriptionConfirmationNeededEvent) from mailman.interfaces.template import ITemplateLoader from mailman.interfaces.user import IUser from mailman.interfaces.usermanager import IUserManager from mailman.interfaces.workflow import IWorkflowStateManager from mailman.utilities.datetime import now from mailman.utilities.string import expand, wrap from public import public from zope.component import getUtility from zope.event import notify from zope.interface import implementer log = logging.getLogger('mailman.subscribe') class WhichSubscriber(Enum): address = 1 user = 2 @implementer(IPendable) class PendableSubscription(dict): PEND_TYPE = 'subscription' @implementer(IPendable) class PendableUnsubscription(dict): PEND_TYPE = 'unsubscription' class _SubscriptionWorkflowCommon(Workflow): """Common support between subscription and unsubscription.""" PENDABLE_CLASS = None def __init__(self, mlist, subscriber): super().__init__() self.mlist = mlist self.address = None self.user = None self.which = None self.member = None self._set_token(TokenOwner.no_one) # The subscriber must be either an IUser or IAddress. if IAddress.providedBy(subscriber): self.address = subscriber self.user = self.address.user self.which = WhichSubscriber.address elif IUser.providedBy(subscriber): self.address = subscriber.preferred_address self.user = subscriber self.which = WhichSubscriber.user self.subscriber = subscriber @property def user_key(self): # For save. return self.user.user_id.hex @user_key.setter def user_key(self, hex_key): # For restore. uid = uuid.UUID(hex_key) self.user = getUtility(IUserManager).get_user_by_id(uid) if self.user is None: self.user = self.address.user @property def address_key(self): # For save. return self.address.email @address_key.setter def address_key(self, email): # For restore. self.address = getUtility(IUserManager).get_address(email) assert self.address is not None @property def subscriber_key(self): return self.which.value @subscriber_key.setter def subscriber_key(self, key): self.which = WhichSubscriber(key) @property def token_owner_key(self): return self.token_owner.value @token_owner_key.setter def token_owner_key(self, value): self.token_owner = TokenOwner(value) def _set_token(self, token_owner): assert isinstance(token_owner, TokenOwner) pendings = getUtility(IPendings) # Clear out the previous pending token if there is one. if self.token is not None: pendings.confirm(self.token) # Create a new token to prevent replay attacks. It seems like this # would produce the same token, but it won't because the pending adds a # bit of randomization. self.token_owner = token_owner if token_owner is TokenOwner.no_one: self.token = None return pendable = self.PENDABLE_CLASS( list_id=self.mlist.list_id, email=self.address.email, display_name=self.address.display_name, when=now().replace(microsecond=0).isoformat(), token_owner=token_owner.name, ) self.token = pendings.add(pendable, timedelta(days=3650)) @public class SubscriptionWorkflow(_SubscriptionWorkflowCommon): """Workflow of a subscription request.""" PENDABLE_CLASS = PendableSubscription INITIAL_STATE = 'sanity_checks' SAVE_ATTRIBUTES = ( 'pre_approved', 'pre_confirmed', 'pre_verified', 'address_key', 'subscriber_key', 'user_key', 'token_owner_key', ) def __init__(self, mlist, subscriber=None, *, pre_verified=False, pre_confirmed=False, pre_approved=False): super().__init__(mlist, subscriber) self.pre_verified = pre_verified self.pre_confirmed = pre_confirmed self.pre_approved = pre_approved def _step_sanity_checks(self): # Ensure that we have both an address and a user, even if the address # is not verified. We can't set the preferred address until it is # verified. if self.user is None: # The address has no linked user so create one, link it, and set # the user's preferred address. assert self.address is not None, 'No address or user' self.user = getUtility(IUserManager).make_user(self.address.email) if self.address is None: assert self.user.preferred_address is None, ( "Preferred address exists, but wasn't used in constructor") addresses = list(self.user.addresses) if len(addresses) == 0: raise AssertionError('User has no addresses: {}'.format( self.user)) # This is rather arbitrary, but we have no choice. self.address = addresses[0] assert self.user is not None and self.address is not None, ( 'Insane sanity check results') # Is this subscriber already a member? if (self.which is WhichSubscriber.user and self.user.preferred_address is not None): subscriber = self.user else: subscriber = self.address if self.mlist.is_subscribed(subscriber): # 2017-04-22 BAW: This branch actually *does* get covered, as I've # verified by a full coverage run, but diffcov for some reason # claims that the test added in the branch that added this code # does not cover the change. That seems like a bug in diffcov. raise AlreadySubscribedError( # pragma: nocover self.mlist.fqdn_listname, self.address.email, MemberRole.member) # Is this email address banned? if IBanManager(self.mlist).is_banned(self.address.email): raise MembershipIsBannedError(self.mlist, self.address.email) # Check if there is already a subscription request for this email. pendings = getUtility(IPendings).find( mlist=self.mlist, pend_type='subscription') for token, pendable in pendings: if pendable['email'] == self.address.email: raise SubscriptionPendingError(self.mlist, self.address.email) # Start out with the subscriber being the token owner. self.push('verification_checks') def _step_verification_checks(self): # Is the address already verified, or is the pre-verified flag set? if self.address.verified_on is None: if self.pre_verified: self.address.verified_on = now() else: # The address being subscribed is not yet verified, so we need # to send a validation email that will also confirm that the # user wants to be subscribed to this mailing list. self.push('send_confirmation') return self.push('confirmation_checks') def _step_confirmation_checks(self): # If the list's subscription policy is open, then the user can be # subscribed right here and now. if self.mlist.subscription_policy is SubscriptionPolicy.open: self.push('do_subscription') return # If we do not need the user's confirmation, then skip to the # moderation checks. if self.mlist.subscription_policy is SubscriptionPolicy.moderate: self.push('moderation_checks') return # If the subscription has been pre-confirmed, then we can skip the # confirmation check can be skipped. If moderator approval is # required we need to check that, otherwise we can go straight to # subscription. if self.pre_confirmed: next_step = ( 'moderation_checks' if self.mlist.subscription_policy is SubscriptionPolicy.confirm_then_moderate # noqa: E131 else 'do_subscription') self.push(next_step) return # The user must confirm their subscription. self.push('send_confirmation') def _step_moderation_checks(self): # Does the moderator need to approve the subscription request? assert self.mlist.subscription_policy in ( SubscriptionPolicy.moderate, SubscriptionPolicy.confirm_then_moderate, ), self.mlist.subscription_policy if self.pre_approved: self.push('do_subscription') else: self.push('get_moderator_approval') def _step_get_moderator_approval(self): # Here's the next step in the workflow, assuming the moderator # approves of the subscription. If they don't, the workflow and # subscription request will just be thrown away. self._set_token(TokenOwner.moderator) self.push('subscribe_from_restored') self.save() log.info('{}: held subscription request from {}'.format( self.mlist.fqdn_listname, self.address.email)) # Possibly send a notification to the list moderators. if self.mlist.admin_immed_notify: subject = _( 'New subscription request to $self.mlist.display_name ' 'from $self.address.email') username = formataddr( (self.subscriber.display_name, self.address.email)) template = getUtility(ITemplateLoader).get( 'list:admin:action:subscribe', self.mlist) text = wrap(expand(template, self.mlist, dict( member=username, ))) # This message should appear to come from the -owner so as # to avoid any useless bounce processing. msg = UserNotification( self.mlist.owner_address, self.mlist.owner_address, subject, text, self.mlist.preferred_language) msg.send(self.mlist) # The workflow must stop running here. raise StopIteration def _step_subscribe_from_restored(self): # Prevent replay attacks. self._set_token(TokenOwner.no_one) # Restore a little extra state that can't be stored in the database # (because the order of setattr() on restore is indeterminate), then # subscribe the user. if self.which is WhichSubscriber.address: self.subscriber = self.address else: assert self.which is WhichSubscriber.user self.subscriber = self.user self.push('do_subscription') def _step_do_subscription(self): # We can immediately subscribe the user to the mailing list. self.member = self.mlist.subscribe(self.subscriber) assert self.token is None and self.token_owner is TokenOwner.no_one, ( 'Unexpected active token at end of subscription workflow') def _step_send_confirmation(self): self._set_token(TokenOwner.subscriber) self.push('do_confirm_verify') self.save() # Triggering this event causes the confirmation message to be sent. notify(SubscriptionConfirmationNeededEvent( self.mlist, self.token, self.address.email)) # Now we wait for the confirmation. raise StopIteration def _step_do_confirm_verify(self): # Restore a little extra state that can't be stored in the database # (because the order of setattr() on restore is indeterminate), then # continue with the confirmation/verification step. if self.which is WhichSubscriber.address: self.subscriber = self.address else: assert self.which is WhichSubscriber.user self.subscriber = self.user # Reset the token so it can't be used in a replay attack. self._set_token(TokenOwner.no_one) # The user has confirmed their subscription request, and also verified # their email address if necessary. This latter needs to be set on the # IAddress, but there's nothing more to do about the confirmation step. # We just continue along with the workflow. if self.address.verified_on is None: self.address.verified_on = now() # The next step depends on the mailing list's subscription policy. next_step = ('moderation_checks' if self.mlist.subscription_policy in ( SubscriptionPolicy.moderate, SubscriptionPolicy.confirm_then_moderate, ) else 'do_subscription') self.push(next_step) @public class UnSubscriptionWorkflow(_SubscriptionWorkflowCommon): """Workflow of a unsubscription request.""" PENDABLE_CLASS = PendableUnsubscription INITIAL_STATE = 'subscription_checks' SAVE_ATTRIBUTES = ( 'pre_approved', 'pre_confirmed', 'address_key', 'user_key', 'subscriber_key', 'token_owner_key', ) def __init__(self, mlist, subscriber=None, *, pre_approved=False, pre_confirmed=False): super().__init__(mlist, subscriber) if IAddress.providedBy(subscriber) or IUser.providedBy(subscriber): self.member = self.mlist.regular_members.get_member( self.address.email) self.pre_confirmed = pre_confirmed self.pre_approved = pre_approved def _step_subscription_checks(self): assert self.mlist.is_subscribed(self.subscriber) self.push('confirmation_checks') def _step_confirmation_checks(self): # If list's unsubscription policy is open, the user can unsubscribe # right now. if self.mlist.unsubscription_policy is SubscriptionPolicy.open: self.push('do_unsubscription') return # If we don't need the user's confirmation, then skip to the moderation # checks. if self.mlist.unsubscription_policy is SubscriptionPolicy.moderate: self.push('moderation_checks') return # If the request is pre-confirmed, then the user can unsubscribe right # now. if self.pre_confirmed: self.push('do_unsubscription') return # The user must confirm their un-subsbcription. self.push('send_confirmation') def _step_send_confirmation(self): self._set_token(TokenOwner.subscriber) self.push('do_confirm_verify') self.save() notify(UnsubscriptionConfirmationNeededEvent( self.mlist, self.token, self.address.email)) raise StopIteration def _step_moderation_checks(self): # Does the moderator need to approve the unsubscription request? assert self.mlist.unsubscription_policy in ( SubscriptionPolicy.moderate, SubscriptionPolicy.confirm_then_moderate, ), self.mlist.unsubscription_policy if self.pre_approved: self.push('do_unsubscription') else: self.push('get_moderator_approval') def _step_get_moderator_approval(self): self._set_token(TokenOwner.moderator) self.push('unsubscribe_from_restored') self.save() log.info('{}: held unsubscription request from {}'.format( self.mlist.fqdn_listname, self.address.email)) if self.mlist.admin_immed_notify: subject = _( 'New unsubscription request to $self.mlist.display_name ' 'from $self.address.email') username = formataddr( (self.subscriber.display_name, self.address.email)) template = getUtility(ITemplateLoader).get( 'list:admin:action:unsubscribe', self.mlist) text = wrap(expand(template, self.mlist, dict( member=username, ))) # This message should appear to come from the -owner so as # to avoid any useless bounce processing. msg = UserNotification( self.mlist.owner_address, self.mlist.owner_address, subject, text, self.mlist.preferred_language) msg.send(self.mlist) # The workflow must stop running here raise StopIteration def _step_do_confirm_verify(self): # Restore a little extra state that can't be stored in the database # (because the order of setattr() on restore is indeterminate), then # continue with the confirmation/verification step. if self.which is WhichSubscriber.address: self.subscriber = self.address else: assert self.which is WhichSubscriber.user self.subscriber = self.user # Reset the token so it can't be used in a replay attack. self._set_token(TokenOwner.no_one) # Restore the member object. self.member = self.mlist.regular_members.get_member(self.address.email) # It's possible the member was already unsubscribed while we were # waiting for the confirmation. if self.member is None: return # The user has confirmed their unsubscription request next_step = ('moderation_checks' if self.mlist.unsubscription_policy in ( SubscriptionPolicy.moderate, SubscriptionPolicy.confirm_then_moderate, ) else 'do_unsubscription') self.push(next_step) def _step_do_unsubscription(self): try: delete_member(self.mlist, self.address.email) except NotAMemberError: # The member has already been unsubscribed. pass self.member = None assert self.token is None and self.token_owner is TokenOwner.no_one, ( 'Unexpected active token at end of subscription workflow') def _step_unsubscribe_from_restored(self): # Prevent replay attacks. self._set_token(TokenOwner.no_one) if self.which is WhichSubscriber.address: self.subscriber = self.address else: assert self.which is WhichSubscriber.user self.subscriber = self.user self.push('do_unsubscription') @public @implementer(ISubscriptionManager) class SubscriptionManager: def __init__(self, mlist): self._mlist = mlist def register(self, subscriber=None, *, pre_verified=False, pre_confirmed=False, pre_approved=False): """See `ISubscriptionManager`.""" workflow = SubscriptionWorkflow( self._mlist, subscriber, pre_verified=pre_verified, pre_confirmed=pre_confirmed, pre_approved=pre_approved) list(workflow) return workflow.token, workflow.token_owner, workflow.member def unregister(self, subscriber=None, *, pre_confirmed=False, pre_approved=False): workflow = UnSubscriptionWorkflow( self._mlist, subscriber, pre_confirmed=pre_confirmed, pre_approved=pre_approved) list(workflow) return workflow.token, workflow.token_owner, workflow.member def confirm(self, token): if token is None: raise LookupError pendable = getUtility(IPendings).confirm(token, expunge=False) if pendable is None: raise LookupError workflow_type = pendable.get('type') assert workflow_type in (PendableSubscription.PEND_TYPE, PendableUnsubscription.PEND_TYPE) workflow = (SubscriptionWorkflow if workflow_type == PendableSubscription.PEND_TYPE else UnSubscriptionWorkflow)(self._mlist) workflow.token = token workflow.restore() # In order to just run the whole workflow, all we need to do # is iterate over the workflow object. On calling the __next__ # over the workflow iterator it automatically executes the steps # that needs to be done. list(workflow) return workflow.token, workflow.token_owner, workflow.member def discard(self, token): with flush(): getUtility(IPendings).confirm(token) getUtility(IWorkflowStateManager).discard(token) def _handle_confirmation_needed_events(event, template_name): subject = 'confirm {}'.format(event.token) confirm_address = event.mlist.confirm_address(event.token) email_address = event.email # Send a verification email to the address. template = getUtility(ITemplateLoader).get(template_name, event.mlist) text = expand(template, event.mlist, dict( token=event.token, subject=subject, confirm_email=confirm_address, user_email=email_address, # For backward compatibility. confirm_address=confirm_address, email_address=email_address, domain_name=event.mlist.domain.mail_host, contact_address=event.mlist.owner_address, )) msg = UserNotification( email_address, confirm_address, subject, text, event.mlist.preferred_language) msg.send(event.mlist, add_precedence=False) @public def handle_SubscriptionConfirmationNeededEvent(event): if not isinstance(event, SubscriptionConfirmationNeededEvent): return _handle_confirmation_needed_events(event, 'list:user:action:subscribe') @public def handle_UnsubscriptionConfirmationNeededEvent(event): if not isinstance(event, UnsubscriptionConfirmationNeededEvent): return _handle_confirmation_needed_events(event, 'list:user:action:unsubscribe') @public def handle_ListDeletingEvent(event): """Delete a mailing list's members when the list is being deleted.""" if not isinstance(event, ListDeletingEvent): return # Find all the members still associated with the mailing list. members = getUtility(ISubscriptionService).find_members( list_id=event.mailing_list.list_id) for member in members: member.unsubscribe() mailman-3.2.2/src/mailman/app/tests/0000755000175000017500000000000013445614541020410 5ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/app/tests/__init__.py0000644000175000017500000000000013244427337022511 0ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/app/tests/test_bounces.py0000644000175000017500000004772113442110351023455 0ustar maxkingmaxking00000000000000# Copyright (C) 2011-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Testing app.bounces functions.""" import os import uuid import shutil import tempfile import unittest from mailman.app.bounces import ( ProbeVERP, StandardVERP, bounce_message, maybe_forward, send_probe) from mailman.app.lifecycle import create_list from mailman.config import config from mailman.interfaces.bounce import UnrecognizedBounceDisposition from mailman.interfaces.languages import ILanguageManager from mailman.interfaces.member import MemberRole from mailman.interfaces.pending import IPendings from mailman.interfaces.usermanager import IUserManager from mailman.testing.helpers import ( LogFileMark, get_queue_messages, specialized_message_from_string as mfs, subscribe) from mailman.testing.layers import ConfigLayer from zope.component import getUtility class TestVERP(unittest.TestCase): """Test header VERP detection.""" layer = ConfigLayer def setUp(self): self._mlist = create_list('test@example.com') self._verper = StandardVERP() def test_no_verp(self): # The empty set is returned when there is no VERP headers. msg = mfs("""\ From: postmaster@example.com To: mailman-bounces@example.com """) self.assertEqual(self._verper.get_verp(self._mlist, msg), set()) def test_verp_in_to(self): # A VERP address is found in the To header. msg = mfs("""\ From: postmaster@example.com To: test-bounces+anne=example.org@example.com """) self.assertEqual(self._verper.get_verp(self._mlist, msg), set(['anne@example.org'])) def test_verp_in_delivered_to(self): # A VERP address is found in the Delivered-To header. msg = mfs("""\ From: postmaster@example.com Delivered-To: test-bounces+anne=example.org@example.com """) self.assertEqual(self._verper.get_verp(self._mlist, msg), set(['anne@example.org'])) def test_verp_in_envelope_to(self): # A VERP address is found in the Envelope-To header. msg = mfs("""\ From: postmaster@example.com Envelope-To: test-bounces+anne=example.org@example.com """) self.assertEqual(self._verper.get_verp(self._mlist, msg), set(['anne@example.org'])) def test_verp_in_apparently_to(self): # A VERP address is found in the Apparently-To header. msg = mfs("""\ From: postmaster@example.com Apparently-To: test-bounces+anne=example.org@example.com """) self.assertEqual(self._verper.get_verp(self._mlist, msg), set(['anne@example.org'])) def test_verp_with_empty_header(self): # A VERP address is found, but there's an empty header. msg = mfs("""\ From: postmaster@example.com To: test-bounces+anne=example.org@example.com To: """) self.assertEqual(self._verper.get_verp(self._mlist, msg), set(['anne@example.org'])) def test_no_verp_with_empty_header(self): # There's an empty header, and no VERP address is found. msg = mfs("""\ From: postmaster@example.com To: """) self.assertEqual(self._verper.get_verp(self._mlist, msg), set()) def test_verp_with_non_match(self): # A VERP address is found, but a header had a non-matching pattern. msg = mfs("""\ From: postmaster@example.com To: test-bounces+anne=example.org@example.com To: test-bounces@example.com """) self.assertEqual(self._verper.get_verp(self._mlist, msg), set(['anne@example.org'])) def test_no_verp_with_non_match(self): # No VERP address is found, and a header had a non-matching pattern. msg = mfs("""\ From: postmaster@example.com To: test-bounces@example.com """) self.assertEqual(self._verper.get_verp(self._mlist, msg), set()) def test_multiple_verps(self): # More than one VERP address was found in the same header. msg = mfs("""\ From: postmaster@example.com To: test-bounces+anne=example.org@example.com To: test-bounces+anne=example.org@example.com """) self.assertEqual(self._verper.get_verp(self._mlist, msg), set(['anne@example.org'])) def test_multiple_verps_different_values(self): # More than one VERP address was found in the same header with # different values. msg = mfs("""\ From: postmaster@example.com To: test-bounces+anne=example.org@example.com To: test-bounces+bart=example.org@example.com """) self.assertEqual(self._verper.get_verp(self._mlist, msg), set(['anne@example.org', 'bart@example.org'])) def test_multiple_verps_different_values_different_headers(self): # More than one VERP address was found in different headers with # different values. msg = mfs("""\ From: postmaster@example.com To: test-bounces+anne=example.org@example.com Apparently-To: test-bounces+bart=example.org@example.com """) self.assertEqual(self._verper.get_verp(self._mlist, msg), set(['anne@example.org', 'bart@example.org'])) class TestSendProbe(unittest.TestCase): """Test sending of the probe message.""" layer = ConfigLayer maxDiff = None def setUp(self): self._mlist = create_list('test@example.com') self._mlist.send_welcome_message = False self._member = subscribe(self._mlist, 'Anne', email='anne@example.com') self._msg = mfs("""\ From: bouncer@example.com To: anne@example.com Subject: You bounced Message-ID: """) def test_token(self): # Show that send_probe() returns a proper token, and that the token # corresponds to a record in the pending database. token = send_probe(self._member, self._msg) pendable = getUtility(IPendings).confirm(token) self.assertEqual(len(pendable.items()), 3) self.assertEqual(set(pendable.keys()), set(['member_id', 'message_id', 'type'])) # member_ids are pended as unicodes. self.assertEqual(uuid.UUID(hex=pendable['member_id']), self._member.member_id) self.assertEqual(pendable['message_id'], '') def test_probe_is_multipart(self): # The probe is a multipart/mixed with two subparts. send_probe(self._member, self._msg) items = get_queue_messages('virgin', expected_count=1) message = items[0].msg self.assertEqual(message.get_content_type(), 'multipart/mixed') self.assertTrue(message.is_multipart()) self.assertEqual(len(message.get_payload()), 2) def test_probe_sends_one_message(self): # send_probe() places one message in the virgin queue. We start out # with no messages in the queue. get_queue_messages('virgin', expected_count=0) send_probe(self._member, self._msg) get_queue_messages('virgin', expected_count=1) def test_probe_contains_original(self): # Show that send_probe() places a properly formatted message in the # virgin queue. send_probe(self._member, self._msg) items = get_queue_messages('virgin', expected_count=1) rfc822 = items[0].msg.get_payload(1) self.assertEqual(rfc822.get_content_type(), 'message/rfc822') self.assertTrue(rfc822.is_multipart()) self.assertEqual(len(rfc822.get_payload()), 1) self.assertEqual(rfc822.get_payload(0).as_string(), self._msg.as_string()) def test_notice(self): # Test that the notice in the first subpart is correct. send_probe(self._member, self._msg) items = get_queue_messages('virgin', expected_count=1) message = items[0].msg notice = message.get_payload(0) self.assertEqual(notice.get_content_type(), 'text/plain') # The interesting bits are the parts that have been interpolated into # the message. For now the best we can do is know that the # interpolation values appear in the message. self.assertMultiLineEqual(notice.get_payload(), """\ This is a probe message. You can ignore this message. The test@example.com mailing list has received a number of bounces from you, indicating that there may be a problem delivering messages to anne@example.com. A sample is attached below. Please examine this message to make sure there are no problems with your email address. You may want to check with your mail administrator for more help. You don't need to do anything to remain an enabled member of the mailing list. If you have any questions or problems, you can contact the mailing list owner at test-owner@example.com """) def test_headers(self): # Check the headers of the outer message. token = send_probe(self._member, self._msg) items = get_queue_messages('virgin', expected_count=1) message = items[0].msg self.assertEqual(message['from'], 'test-bounces+{0}@example.com'.format(token)) self.assertEqual(message['to'], 'anne@example.com') self.assertEqual(message['subject'], 'Test mailing list probe message') def test_no_precedence_header(self): # Probe messages should not have a Precedence header (LP: #808821). send_probe(self._member, self._msg) items = get_queue_messages('virgin', expected_count=1) self.assertIsNone(items[0].msg['precedence']) class TestSendProbeNonEnglish(unittest.TestCase): """Test sending of the probe message to a non-English speaker.""" layer = ConfigLayer maxDiff = None def setUp(self): self._mlist = create_list('test@example.com') self._member = subscribe(self._mlist, 'Anne', email='anne@example.com') self._msg = mfs("""\ From: bouncer@example.com To: anne@example.com Subject: You bounced Message-ID: """) # Set up the translation context. self._var_dir = tempfile.mkdtemp() self.addCleanup(shutil.rmtree, self._var_dir) xx_template_path = os.path.join( self._var_dir, 'templates', 'site', 'xx', 'list:user:notice:probe.txt') os.makedirs(os.path.dirname(xx_template_path)) config.push('xx template dir', """\ [paths.testing] var_dir: {} """.format(self._var_dir)) self.addCleanup(config.pop, 'xx template dir') language_manager = getUtility(ILanguageManager) language_manager.add('xx', 'utf-8', 'Freedonia') self._member.preferences.preferred_language = 'xx' with open(xx_template_path, 'w') as fp: print("""\ blah blah blah $listname $address $owneraddr """, file=fp) def test_subject_with_member_nonenglish(self): # Test that members with non-English preferred language get a Subject # header in the expected language. send_probe(self._member, self._msg) items = get_queue_messages('virgin', expected_count=1) self.assertEqual( items[0].msg['subject'].encode(), '=?utf-8?q?ailing-may_ist-lay_Test_obe-pray_essage-may?=') def test_probe_notice_with_member_nonenglish(self): # Test that a member with non-English preferred language gets the # probe message in their language. send_probe(self._member, self._msg) items = get_queue_messages('virgin', expected_count=1) message = items[0].msg notice = message.get_payload(0).get_payload() self.assertMultiLineEqual(notice, """\ blah blah blah test@example.com anne@example.com test-owner@example.com """) class TestProbe(unittest.TestCase): """Test VERP probe parsing.""" layer = ConfigLayer def setUp(self): self._mlist = create_list('test@example.com') self._mlist.send_welcome_message = False self._member = subscribe(self._mlist, 'Anne', email='anne@example.com') self._msg = mfs("""\ From: bouncer@example.com To: anne@example.com Subject: You bounced Message-ID: """) def test_get_addresses(self): # Be able to extract the probed address from the pending database # based on the token in a probe bounce. token = send_probe(self._member, self._msg) # Simulate a bounce of the message in the virgin queue. items = get_queue_messages('virgin', expected_count=1) message = items[0].msg bounce = mfs("""\ To: {0} From: mail-daemon@example.com """.format(message['From'])) addresses = ProbeVERP().get_verp(self._mlist, bounce) self.assertEqual(addresses, set(['anne@example.com'])) # The pendable is no longer in the database. self.assertIsNone(getUtility(IPendings).confirm(token)) class TestMaybeForward(unittest.TestCase): """Test forwarding of unrecognized bounces.""" layer = ConfigLayer maxDiff = None def setUp(self): config.push('test config', """ [mailman] site_owner: postmaster@example.com """) self.addCleanup(config.pop, 'test config') self._mlist = create_list('test@example.com') self._mlist.send_welcome_message = False self._msg = mfs("""\ From: bouncer@example.com To: test-bounces@example.com Subject: You bounced Message-ID: """) def test_maybe_forward_discard(self): # When forward_unrecognized_bounces_to is set to discard, no bounce # messages are forwarded. self._mlist.forward_unrecognized_bounces_to = ( UnrecognizedBounceDisposition.discard) # The only artifact of this call is a log file entry. mark = LogFileMark('mailman.bounce') maybe_forward(self._mlist, self._msg) get_queue_messages('virgin', expected_count=0) line = mark.readline() self.assertEqual( line[-40:-1], 'Discarding unrecognized bounce: ') def test_maybe_forward_list_owner(self): # Set up some owner and moderator addresses. user_manager = getUtility(IUserManager) anne = user_manager.create_address('anne@example.com') bart = user_manager.create_address('bart@example.com') cris = user_manager.create_address('cris@example.com') dave = user_manager.create_address('dave@example.com') # Regular members. elle = user_manager.create_address('elle@example.com') fred = user_manager.create_address('fred@example.com') self._mlist.subscribe(anne, MemberRole.owner) self._mlist.subscribe(bart, MemberRole.owner) self._mlist.subscribe(cris, MemberRole.moderator) self._mlist.subscribe(dave, MemberRole.moderator) self._mlist.subscribe(elle, MemberRole.member) self._mlist.subscribe(fred, MemberRole.member) # When forward_unrecognized_bounces_to is set to owners, the # bounce is forwarded to the list owners and moderators. self._mlist.forward_unrecognized_bounces_to = ( UnrecognizedBounceDisposition.administrators) maybe_forward(self._mlist, self._msg) items = get_queue_messages('virgin', expected_count=1) msg = items[0].msg self.assertEqual(msg['subject'], 'Uncaught bounce notification') self.assertEqual(msg['from'], 'postmaster@example.com') self.assertEqual(msg['to'], 'test-owner@example.com') # The first attachment is a notification message with a url. payload = msg.get_payload(0) self.assertEqual(payload.get_content_type(), 'text/plain') body = payload.get_payload() self.assertMultiLineEqual(body, """\ The attached message was received as a bounce, but either the bounce format was not recognized, or no member addresses could be extracted from it. This mailing list has been configured to send all unrecognized bounce messages to the list administrator(s). """) # The second attachment should be a message/rfc822 containing the # original bounce message. payload = msg.get_payload(1) self.assertEqual(payload.get_content_type(), 'message/rfc822') bounce = payload.get_payload(0) self.assertEqual(bounce.as_string(), self._msg.as_string()) # All of the owners and moderators, but none of the members, should be # recipients of this message. self.assertEqual(items[0].msgdata['recipients'], set(['anne@example.com', 'bart@example.com', 'cris@example.com', 'dave@example.com'])) def test_maybe_forward_site_owner(self): # Set up some owner and moderator addresses. user_manager = getUtility(IUserManager) anne = user_manager.create_address('anne@example.com') bart = user_manager.create_address('bart@example.com') cris = user_manager.create_address('cris@example.com') dave = user_manager.create_address('dave@example.com') # Regular members. elle = user_manager.create_address('elle@example.com') fred = user_manager.create_address('fred@example.com') self._mlist.subscribe(anne, MemberRole.owner) self._mlist.subscribe(bart, MemberRole.owner) self._mlist.subscribe(cris, MemberRole.moderator) self._mlist.subscribe(dave, MemberRole.moderator) self._mlist.subscribe(elle, MemberRole.member) self._mlist.subscribe(fred, MemberRole.member) # When forward_unrecognized_bounces_to is set to owners, the # bounce is forwarded to the list owners and moderators. self._mlist.forward_unrecognized_bounces_to = ( UnrecognizedBounceDisposition.site_owner) maybe_forward(self._mlist, self._msg) items = get_queue_messages('virgin', expected_count=1) msg = items[0].msg self.assertEqual(msg['subject'], 'Uncaught bounce notification') self.assertEqual(msg['from'], 'postmaster@example.com') self.assertEqual(msg['to'], 'postmaster@example.com') # The first attachment is a notification message with a url. payload = msg.get_payload(0) self.assertEqual(payload.get_content_type(), 'text/plain') body = payload.get_payload() self.assertMultiLineEqual(body, """\ The attached message was received as a bounce, but either the bounce format was not recognized, or no member addresses could be extracted from it. This mailing list has been configured to send all unrecognized bounce messages to the list administrator(s). """) # The second attachment should be a message/rfc822 containing the # original bounce message. payload = msg.get_payload(1) self.assertEqual(payload.get_content_type(), 'message/rfc822') bounce = payload.get_payload(0) self.assertEqual(bounce.as_string(), self._msg.as_string()) # All of the owners and moderators, but none of the members, should be # recipients of this message. self.assertEqual(items[0].msgdata['recipients'], set(['postmaster@example.com'])) class TestBounceMessage(unittest.TestCase): """Test the `mailman.app.bounces.bounce_message()` function.""" layer = ConfigLayer def setUp(self): self._mlist = create_list('test@example.com') self._msg = mfs("""\ From: anne@example.com To: test@example.com Subject: Ignore """) def test_no_sender(self): # The message won't be bounced if it has no discernible sender. del self._msg['from'] bounce_message(self._mlist, self._msg) # Nothing in the virgin queue means nothing's been bounced. get_queue_messages('virgin', expected_count=0) mailman-3.2.2/src/mailman/app/tests/test_digests.py0000644000175000017500000002346213442110351023455 0ustar maxkingmaxking00000000000000# Copyright (C) 2015-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Digest helper tests.""" import os import unittest from datetime import timedelta from mailman.app.digests import ( bump_digest_number_and_volume, maybe_send_digest_now) from mailman.app.lifecycle import create_list from mailman.config import config from mailman.interfaces.digests import DigestFrequency from mailman.interfaces.member import DeliveryMode from mailman.runners.digest import DigestRunner from mailman.testing.helpers import ( get_queue_messages, make_testable_runner, specialized_message_from_string as mfs, subscribe) from mailman.testing.layers import ConfigLayer from mailman.utilities.datetime import factory, now as right_now class TestBumpDigest(unittest.TestCase): layer = ConfigLayer def setUp(self): self._mlist = create_list('ant@example.com') self._mlist.volume = 7 self._mlist.next_digest_number = 4 self.right_now = right_now() def test_bump_no_previous_digest(self): self._mlist.digest_last_sent_at = None bump_digest_number_and_volume(self._mlist) self.assertEqual(self._mlist.volume, 7) self.assertEqual(self._mlist.next_digest_number, 5) self.assertEqual(self._mlist.digest_last_sent_at, self.right_now) def test_bump_yearly(self): self._mlist.digest_last_sent_at = self.right_now + timedelta( days=-370) self._mlist.digest_volume_frequency = DigestFrequency.yearly bump_digest_number_and_volume(self._mlist) self.assertEqual(self._mlist.volume, 8) self.assertEqual(self._mlist.next_digest_number, 1) self.assertEqual(self._mlist.digest_last_sent_at, self.right_now) def test_bump_yearly_not_yet(self): self._mlist.digest_last_sent_at = self.right_now + timedelta( days=-200) self._mlist.digest_volume_frequency = DigestFrequency.yearly bump_digest_number_and_volume(self._mlist) self.assertEqual(self._mlist.volume, 7) self.assertEqual(self._mlist.next_digest_number, 5) self.assertEqual(self._mlist.digest_last_sent_at, self.right_now) def test_bump_monthly(self): self._mlist.digest_last_sent_at = self.right_now + timedelta( days=-32) self._mlist.digest_volume_frequency = DigestFrequency.monthly bump_digest_number_and_volume(self._mlist) self.assertEqual(self._mlist.volume, 8) self.assertEqual(self._mlist.next_digest_number, 1) self.assertEqual(self._mlist.digest_last_sent_at, self.right_now) def test_bump_monthly_not_yet(self): # The normal test date starts on the first day of the month, so let's # fast forward it a few days so we can set the digest last sent time # to earlier in the same month. self._mlist.digest_last_sent_at = self.right_now factory.fast_forward(days=26) self._mlist.digest_volume_frequency = DigestFrequency.monthly bump_digest_number_and_volume(self._mlist) self.assertEqual(self._mlist.volume, 7) self.assertEqual(self._mlist.next_digest_number, 5) self.assertEqual(self._mlist.digest_last_sent_at, right_now()) def test_bump_quarterly(self): self._mlist.digest_last_sent_at = self.right_now + timedelta( days=-93) self._mlist.digest_volume_frequency = DigestFrequency.quarterly bump_digest_number_and_volume(self._mlist) self.assertEqual(self._mlist.volume, 8) self.assertEqual(self._mlist.next_digest_number, 1) self.assertEqual(self._mlist.digest_last_sent_at, self.right_now) def test_bump_quarterly_not_yet(self): self._mlist.digest_last_sent_at = self.right_now + timedelta( days=-88) self._mlist.digest_volume_frequency = DigestFrequency.quarterly bump_digest_number_and_volume(self._mlist) self.assertEqual(self._mlist.volume, 7) self.assertEqual(self._mlist.next_digest_number, 5) self.assertEqual(self._mlist.digest_last_sent_at, self.right_now) def test_bump_weekly(self): self._mlist.digest_last_sent_at = self.right_now + timedelta( days=-8) self._mlist.digest_volume_frequency = DigestFrequency.weekly bump_digest_number_and_volume(self._mlist) self.assertEqual(self._mlist.volume, 8) self.assertEqual(self._mlist.next_digest_number, 1) self.assertEqual(self._mlist.digest_last_sent_at, self.right_now) def test_bump_weekly_not_yet(self): # The normal test date starts on the first day of the week, so let's # fast forward it a few days so we can set the digest last sent time # to earlier in the same week. self._mlist.digest_last_sent_at = self.right_now factory.fast_forward(days=3) self._mlist.digest_volume_frequency = DigestFrequency.weekly bump_digest_number_and_volume(self._mlist) self.assertEqual(self._mlist.volume, 7) self.assertEqual(self._mlist.next_digest_number, 5) self.assertEqual(self._mlist.digest_last_sent_at, right_now()) def test_bump_daily(self): self._mlist.digest_last_sent_at = self.right_now + timedelta( hours=-27) self._mlist.digest_volume_frequency = DigestFrequency.daily bump_digest_number_and_volume(self._mlist) self.assertEqual(self._mlist.volume, 8) self.assertEqual(self._mlist.next_digest_number, 1) self.assertEqual(self._mlist.digest_last_sent_at, self.right_now) def test_bump_daily_not_yet(self): self._mlist.digest_last_sent_at = self.right_now + timedelta( hours=-5) self._mlist.digest_volume_frequency = DigestFrequency.daily bump_digest_number_and_volume(self._mlist) self.assertEqual(self._mlist.volume, 7) self.assertEqual(self._mlist.next_digest_number, 5) self.assertEqual(self._mlist.digest_last_sent_at, self.right_now) def test_bump_bad_frequency(self): self._mlist.digest_last_sent_at = self.right_now + timedelta( hours=-22) self._mlist.digest_volume_frequency = -10 self.assertRaises(AssertionError, bump_digest_number_and_volume, self._mlist) class TestMaybeSendDigest(unittest.TestCase): layer = ConfigLayer def setUp(self): self._mlist = create_list('ant@example.com') self._mlist.send_welcome_message = False self._mailbox_path = os.path.join(self._mlist.data_path, 'digest.mmdf') # The mailing list needs at least one digest recipient. member = subscribe(self._mlist, 'Anne') member.preferences.delivery_mode = DeliveryMode.plaintext_digests self._subject_number = 1 self._runner = make_testable_runner(DigestRunner, 'digest') def _to_digest(self, count=1): for i in range(count): msg = mfs("""\ To: ant@example.com From: anne@example.com Subject: message {} """.format(self._subject_number)) self._subject_number += 1 config.handlers['to-digest'].process(self._mlist, msg, {}) def test_send_digest_over_threshold(self): # Put a few messages in the digest. self._to_digest(3) # Set the size threshold low enough to trigger a send. self._mlist.digest_size_threshold = 0.1 maybe_send_digest_now(self._mlist) self._runner.run() # There are no digests in flight now, and a single digest message has # been sent. get_queue_messages('digest', expected_count=0) self.assertFalse(os.path.exists(self._mailbox_path)) items = get_queue_messages('virgin', expected_count=1) digest_contents = str(items[0].msg) self.assertIn('Subject: message 1', digest_contents) self.assertIn('Subject: message 2', digest_contents) def test_dont_send_digest_under_threshold(self): # Put a few messages in the digest. self._to_digest(3) # Set the size threshold high enough to not trigger a send. self._mlist.digest_size_threshold = 100 maybe_send_digest_now(self._mlist) self._runner.run() # A digest is still being collected, but none have been sent. get_queue_messages('digest', expected_count=0) self.assertGreater(os.path.getsize(self._mailbox_path), 0) self.assertLess(os.path.getsize(self._mailbox_path), 100 * 1024.0) get_queue_messages('virgin', expected_count=0) def test_force_send_digest_under_threshold(self): # Put a few messages in the digest. self._to_digest(3) # Set the size threshold high enough to not trigger a send. self._mlist.digest_size_threshold = 100 # Force sending a digest anyway. maybe_send_digest_now(self._mlist, force=True) self._runner.run() # There are no digests in flight now, and a single digest message has # been sent. get_queue_messages('digest', expected_count=0) self.assertFalse(os.path.exists(self._mailbox_path)) items = get_queue_messages('virgin', expected_count=1) digest_contents = str(items[0].msg) self.assertIn('Subject: message 1', digest_contents) self.assertIn('Subject: message 2', digest_contents) mailman-3.2.2/src/mailman/app/tests/test_inject.py0000644000175000017500000002523513442110351023267 0ustar maxkingmaxking00000000000000# Copyright (C) 2011-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Testing app.inject functions.""" import unittest from mailman.app.inject import inject_message, inject_text from mailman.app.lifecycle import create_list from mailman.email.message import Message from mailman.testing.helpers import ( get_queue_messages, specialized_message_from_string as message_from_string) from mailman.testing.layers import ConfigLayer NL = '\n' class TestInjectMessage(unittest.TestCase): """Test message injection.""" layer = ConfigLayer def setUp(self): self.mlist = create_list('test@example.com') self.msg = message_from_string("""\ From: anne@example.com To: test@example.com Subject: A test message Message-ID: Date: Tue, 14 Jun 2011 21:12:00 -0400 Nothing. """) # Let assertMultiLineEqual work without bounds. def test_inject_message(self): # Test basic inject_message() call. inject_message(self.mlist, self.msg) items = get_queue_messages('in', expected_count=1) self.assertMultiLineEqual(items[0].msg.as_string(), self.msg.as_string()) self.assertEqual(items[0].msgdata['listid'], 'test.example.com') self.assertEqual(items[0].msgdata['original_size'], len(self.msg.as_string())) def test_inject_message_with_recipients(self): # Explicit recipients end up in the metadata. recipients = ['bart@example.com', 'cris@example.com'] inject_message(self.mlist, self.msg, recipients) items = get_queue_messages('in', expected_count=1) self.assertEqual(items[0].msgdata['recipients'], recipients) def test_inject_message_to_queue(self): # Explicitly use a different queue. inject_message(self.mlist, self.msg, switchboard='virgin') get_queue_messages('in', expected_count=0) items = get_queue_messages('virgin', expected_count=1) self.assertMultiLineEqual(items[0].msg.as_string(), self.msg.as_string()) self.assertEqual(items[0].msgdata['listid'], 'test.example.com') self.assertEqual(items[0].msgdata['original_size'], len(self.msg.as_string())) def test_inject_message_without_message_id(self): # inject_message() adds a Message-ID header if it's missing. del self.msg['message-id'] self.assertNotIn('message-id', self.msg) inject_message(self.mlist, self.msg) self.assertIn('message-id', self.msg) items = get_queue_messages('in', expected_count=1) self.assertIn('message-id', items[0].msg) self.assertEqual(items[0].msg['message-id'], self.msg['message-id']) def test_inject_message_without_date(self): # inject_message() adds a Date header if it's missing. del self.msg['date'] self.assertNotIn('date', self.msg) inject_message(self.mlist, self.msg) self.assertIn('date', self.msg) items = get_queue_messages('in', expected_count=1) self.assertIn('date', items[0].msg) self.assertEqual(items[0].msg['date'], self.msg['date']) def test_inject_message_with_keywords(self): # Keyword arguments are copied into the metadata. inject_message(self.mlist, self.msg, foo='yes', bar='no') items = get_queue_messages('in', expected_count=1) self.assertEqual(items[0].msgdata['foo'], 'yes') self.assertEqual(items[0].msgdata['bar'], 'no') def test_inject_message_id_hash(self): # When the injected message has a Message-ID header, the injected # message will also get an Message-ID-Hash header. inject_message(self.mlist, self.msg) items = get_queue_messages('in', expected_count=1) self.assertEqual(items[0].msg['message-id-hash'], '4CMWUN6BHVCMHMDAOSJZ2Q72G5M32MWB') def test_inject_message_id_hash_without_message_id(self): # When the injected message does not have a Message-ID header, a # Message-ID header will be added, and the injected message will also # get an Message-ID-Hash header. del self.msg['message-id'] self.assertNotIn('message-id', self.msg) self.assertNotIn('message-id-hash', self.msg) inject_message(self.mlist, self.msg) items = get_queue_messages('in', expected_count=1) self.assertIn('message-id', items[0].msg) self.assertIn('message-id-hash', items[0].msg) class TestInjectText(unittest.TestCase): """Test text injection.""" layer = ConfigLayer maxDiff = None def setUp(self): self.mlist = create_list('test@example.com') self.text = """\ From: bart@example.com To: test@example.com Subject: A test message Message-ID: Date: Tue, 14 Jun 2011 21:12:00 -0400 Nothing. """ def _remove_line(self, header): return NL.join(line for line in self.text.splitlines() if not line.lower().startswith(header)) def test_inject_text(self): # Test basic inject_text() call. inject_text(self.mlist, self.text) items = get_queue_messages('in', expected_count=1) self.assertIsInstance(items[0].msg, Message) self.assertEqual(items[0].msg['message-id-hash'], 'GUXXQKNCHBFQAHGBFMGCME6HKZCUUH3K') # Delete these headers because they don't exist in the original text. del items[0].msg['message-id-hash'] del items[0].msg['x-message-id-hash'] self.assertMultiLineEqual(items[0].msg.as_string(), self.text) self.assertEqual(items[0].msgdata['listid'], 'test.example.com') self.assertEqual(items[0].msgdata['original_size'], # Add back the Message-ID-Hash and X-Message-ID-Hash # headers which wer in the message contributing to the # original_size, but weren't in the original text. # Don't forget the space, delimeter, and newline! len(self.text) + 50 + 52) def test_inject_text_with_recipients(self): # Explicit recipients end up in the metadata. recipients = ['bart@example.com', 'cris@example.com'] inject_text(self.mlist, self.text, recipients) items = get_queue_messages('in', expected_count=1) self.assertEqual(items[0].msgdata['recipients'], recipients) def test_inject_text_to_queue(self): # Explicitly use a different queue. inject_text(self.mlist, self.text, switchboard='virgin') get_queue_messages('in', expected_count=0) items = get_queue_messages('virgin', expected_count=1) # Remove the Message-ID-Hash header which isn't in the original text. del items[0].msg['message-id-hash'] del items[0].msg['x-message-id-hash'] self.assertMultiLineEqual(items[0].msg.as_string(), self.text) self.assertEqual(items[0].msgdata['listid'], 'test.example.com') self.assertEqual(items[0].msgdata['original_size'], # Add back the Message-ID-Hash and X-Message-ID-Hash # headers which wer in the message contributing to the # original_size, but weren't in the original text. # Don't forget the space, delimeter, and newline! len(self.text) + 50 + 52) def test_inject_text_without_message_id(self): # inject_text() adds a Message-ID header if it's missing. filtered = self._remove_line('message-id') self.assertNotIn('Message-ID', filtered) inject_text(self.mlist, filtered) items = get_queue_messages('in', expected_count=1) self.assertIn('message-id', items[0].msg) def test_inject_text_without_date(self): # inject_text() adds a Date header if it's missing. filtered = self._remove_line('date') self.assertNotIn('date', filtered) inject_text(self.mlist, self.text) items = get_queue_messages('in', expected_count=1) self.assertIn('date', items[0].msg) def test_inject_text_adds_original_size(self): # The metadata gets an original_size attribute that is the length of # the injected text. inject_text(self.mlist, self.text) items = get_queue_messages('in', expected_count=1) self.assertEqual(items[0].msgdata['original_size'], # Add back the Message-ID-Hash and X-Message-ID-Hash # headers which wer in the message contributing to the # original_size, but weren't in the original text. # Don't forget the space, delimeter, and newline! len(self.text) + 50 + 52) def test_inject_text_with_keywords(self): # Keyword arguments are copied into the metadata. inject_text(self.mlist, self.text, foo='yes', bar='no') items = get_queue_messages('in', expected_count=1) self.assertEqual(items[0].msgdata['foo'], 'yes') self.assertEqual(items[0].msgdata['bar'], 'no') def test_inject_message_id_hash(self): # When the injected message has a Message-ID header, the injected # message will also get an Message-ID-Hash header. inject_text(self.mlist, self.text) items = get_queue_messages('in', expected_count=1) self.assertEqual(items[0].msg['message-id-hash'], 'GUXXQKNCHBFQAHGBFMGCME6HKZCUUH3K') def test_inject_message_id_hash_without_message_id(self): # When the injected message does not have a Message-ID header, a # Message-ID header will be added, and the injected message will also # get an Message-ID-Hash header. filtered = self._remove_line('message-id') self.assertNotIn('Message-ID', filtered) self.assertNotIn('Message-ID-Hash', filtered) inject_text(self.mlist, filtered) items = get_queue_messages('in', expected_count=1) self.assertIn('message-id', items[0].msg) self.assertIn('message-id-hash', items[0].msg) mailman-3.2.2/src/mailman/app/tests/test_lifecycle.py0000644000175000017500000000776213442110351023757 0ustar maxkingmaxking00000000000000# Copyright (C) 2012-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Test the high level list lifecycle API.""" import os import shutil import unittest from mailman.app.lifecycle import ( InvalidListNameError, create_list, remove_list) from mailman.interfaces.address import InvalidEmailAddressError from mailman.interfaces.domain import BadDomainSpecificationError from mailman.interfaces.listmanager import IListManager from mailman.testing.helpers import LogFileMark, configuration from mailman.testing.layers import ConfigLayer from zope.component import getUtility class TestLifecycle(unittest.TestCase): """Test the high level list lifecycle API.""" layer = ConfigLayer def test_posting_address_validation(self): # Creating a mailing list with a bogus address raises an exception. self.assertRaises(InvalidEmailAddressError, create_list, 'bogus address') def test_listname_validation(self): # Creating a mailing list with invalid characters in the listname # raises an exception. self.assertRaises(InvalidListNameError, create_list, 'my/list@example.com') @configuration('mailman', listname_chars=r'[a-z0-9-+\]') def test_bad_config_listname_chars(self): mark = LogFileMark('mailman.error') # This list create should succeed but log an error mlist = create_list('test@example.com') # Check the error log. self.assertRegex( mark.readline(), r'^.*Bad config\.mailman\.listname_chars setting: ' r'\[a-z0-9-\+\\]: ' '(unterminated character set|' 'unexpected end of regular expression)$' ) # Check that the list was actually created. self.assertIs(os.path.isdir(mlist.data_path), True) @configuration('mailman', listname_chars='[a-z]') def test_listname_with_minimal_listname_chars(self): # This only allows letters in the listname. A listname with digits # Raises an exception. self.assertRaises(InvalidListNameError, create_list, 'list1@example.com') def test_unregistered_domain(self): # Creating a list with an unregistered domain raises an exception. self.assertRaises(BadDomainSpecificationError, create_list, 'test@nodomain.example.org') @unittest.skipIf(os.getuid() == 0, 'Cannot run as root') def test_remove_list_error(self): # An error occurs while deleting the list's data directory. mlist = create_list('test@example.com') os.chmod(mlist.data_path, 0) self.addCleanup(shutil.rmtree, mlist.data_path) self.assertRaises(OSError, remove_list, mlist) os.chmod(mlist.data_path, 0o777) def test_create_no_such_style(self): mlist = create_list('ant@example.com', style_name='bogus') # The MailmanList._preferred_language column isn't set so there's no # valid mapping to an ILanguage. Therefore this call will produce a # KeyError. self.assertRaises(KeyError, getattr, mlist, 'preferred_language') def test_remove_list_without_data_path(self): mlist = create_list('ant@example.com') shutil.rmtree(mlist.data_path) remove_list(mlist) self.assertIsNone(getUtility(IListManager).get('ant@example.com')) mailman-3.2.2/src/mailman/app/tests/test_membership.py0000644000175000017500000003023613442110351024143 0ustar maxkingmaxking00000000000000# Copyright (C) 2011-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Tests of application level membership functions.""" import unittest from mailman.app.lifecycle import create_list from mailman.app.membership import add_member, delete_member from mailman.core.constants import system_preferences from mailman.interfaces.bans import IBanManager from mailman.interfaces.member import ( AlreadySubscribedError, DeliveryMode, MemberRole, MembershipIsBannedError, NotAMemberError) from mailman.interfaces.subscriptions import RequestRecord from mailman.interfaces.usermanager import IUserManager from mailman.testing.layers import ConfigLayer from mailman.utilities.datetime import now from zope.component import getUtility class TestAddMember(unittest.TestCase): layer = ConfigLayer def setUp(self): self._mlist = create_list('test@example.com') def test_add_member_new_user(self): # Test subscribing a user to a mailing list when the email address has # not yet been associated with a user. member = add_member( self._mlist, RequestRecord('aperson@example.com', 'Anne Person', DeliveryMode.regular, system_preferences.preferred_language)) self.assertEqual(member.address.email, 'aperson@example.com') self.assertEqual(member.list_id, 'test.example.com') self.assertEqual(member.role, MemberRole.member) def test_add_member_existing_user(self): # Test subscribing a user to a mailing list when the email address has # already been associated with a user. user_manager = getUtility(IUserManager) user_manager.create_user('aperson@example.com', 'Anne Person') member = add_member( self._mlist, RequestRecord('aperson@example.com', 'Anne Person', DeliveryMode.regular, system_preferences.preferred_language)) self.assertEqual(member.address.email, 'aperson@example.com') self.assertEqual(member.list_id, 'test.example.com') def test_add_member_banned(self): # Test that members who are banned by specific address cannot # subscribe to the mailing list. IBanManager(self._mlist).ban('anne@example.com') with self.assertRaises(MembershipIsBannedError) as cm: add_member( self._mlist, RequestRecord('anne@example.com', 'Anne Person', DeliveryMode.regular, system_preferences.preferred_language)) self.assertEqual( str(cm.exception), 'anne@example.com is not allowed to subscribe to test@example.com') def test_add_member_globally_banned(self): # Test that members who are banned by specific address cannot # subscribe to the mailing list. IBanManager(None).ban('anne@example.com') self.assertRaises( MembershipIsBannedError, add_member, self._mlist, RequestRecord('anne@example.com', 'Anne Person', DeliveryMode.regular, system_preferences.preferred_language)) def test_add_member_banned_from_different_list(self): # Test that members who are banned by on a different list can still be # subscribed to other mlists. sample_list = create_list('sample@example.com') IBanManager(sample_list).ban('anne@example.com') member = add_member( self._mlist, RequestRecord('anne@example.com', 'Anne Person', DeliveryMode.regular, system_preferences.preferred_language)) self.assertEqual(member.address.email, 'anne@example.com') def test_add_member_banned_by_pattern(self): # Addresses matching regexp ban patterns cannot subscribe. IBanManager(self._mlist).ban('^.*@example.com') self.assertRaises( MembershipIsBannedError, add_member, self._mlist, RequestRecord('anne@example.com', 'Anne Person', DeliveryMode.regular, system_preferences.preferred_language)) def test_add_member_globally_banned_by_pattern(self): # Addresses matching global regexp ban patterns cannot subscribe. IBanManager(None).ban('^.*@example.com') self.assertRaises( MembershipIsBannedError, add_member, self._mlist, RequestRecord('anne@example.com', 'Anne Person', DeliveryMode.regular, system_preferences.preferred_language)) def test_add_member_banned_from_different_list_by_pattern(self): # Addresses matching regexp ban patterns on one list can still # subscribe to other mailing lists. sample_list = create_list('sample@example.com') IBanManager(sample_list).ban('^.*@example.com') member = add_member( self._mlist, RequestRecord('anne@example.com', 'Anne Person', DeliveryMode.regular, system_preferences.preferred_language)) self.assertEqual(member.address.email, 'anne@example.com') def test_add_member_moderator(self): # Test adding a moderator to a mailing list. member = add_member( self._mlist, RequestRecord('aperson@example.com', 'Anne Person', DeliveryMode.regular, system_preferences.preferred_language), MemberRole.moderator) self.assertEqual(member.address.email, 'aperson@example.com') self.assertEqual(member.list_id, 'test.example.com') self.assertEqual(member.role, MemberRole.moderator) def test_add_member_twice(self): # Adding a member with the same role twice causes an # AlreadySubscribedError to be raised. add_member( self._mlist, RequestRecord('aperson@example.com', 'Anne Person', DeliveryMode.regular, system_preferences.preferred_language), MemberRole.member) with self.assertRaises(AlreadySubscribedError) as cm: add_member( self._mlist, RequestRecord('aperson@example.com', 'Anne Person', DeliveryMode.regular, system_preferences.preferred_language), MemberRole.member) self.assertEqual(cm.exception.fqdn_listname, 'test@example.com') self.assertEqual(cm.exception.email, 'aperson@example.com') self.assertEqual(cm.exception.role, MemberRole.member) def test_add_member_with_different_roles(self): # Adding a member twice with different roles is okay. member_1 = add_member( self._mlist, RequestRecord('aperson@example.com', 'Anne Person', DeliveryMode.regular, system_preferences.preferred_language), MemberRole.member) member_2 = add_member( self._mlist, RequestRecord('aperson@example.com', 'Anne Person', DeliveryMode.regular, system_preferences.preferred_language), MemberRole.owner) self.assertEqual(member_1.list_id, member_2.list_id) self.assertEqual(member_1.address, member_2.address) self.assertEqual(member_1.user, member_2.user) self.assertNotEqual(member_1.member_id, member_2.member_id) self.assertEqual(member_1.role, MemberRole.member) self.assertEqual(member_2.role, MemberRole.owner) def test_add_member_with_mixed_case_email(self): # LP: #1425359 - Mailman is case-perserving, case-insensitive. This # test subscribes the lower case address and ensures the original # mixed case address can't be subscribed. email = 'APerson@example.com' add_member( self._mlist, RequestRecord(email.lower(), 'Ann Person', DeliveryMode.regular, system_preferences.preferred_language)) with self.assertRaises(AlreadySubscribedError) as cm: add_member( self._mlist, RequestRecord(email, 'Ann Person', DeliveryMode.regular, system_preferences.preferred_language)) self.assertEqual(cm.exception.email, email) def test_add_member_with_lower_case_email(self): # LP: #1425359 - Mailman is case-perserving, case-insensitive. This # test subscribes the mixed case address and ensures the lower cased # address can't be added. email = 'APerson@example.com' add_member( self._mlist, RequestRecord(email, 'Ann Person', DeliveryMode.regular, system_preferences.preferred_language)) with self.assertRaises(AlreadySubscribedError) as cm: add_member( self._mlist, RequestRecord(email.lower(), 'Ann Person', DeliveryMode.regular, system_preferences.preferred_language)) self.assertEqual(cm.exception.email, email.lower()) def test_delete_nonmembers_on_adding_member(self): # GL: #237 - When a new address is subscribed, any existing nonmember # subscriptions for this address; or any addresses also controlled by # this user, are deleted. anne_nonmember = add_member( self._mlist, RequestRecord('aperson@example.com', 'Anne Person', DeliveryMode.regular, system_preferences.preferred_language), MemberRole.nonmember) # Add a few other validated addresses to this user, and subscribe them # as nonmembers. for email in ('anne.person@example.com', 'a.person@example.com'): address = anne_nonmember.user.register(email) address.verified_on = now() self._mlist.subscribe(address, MemberRole.nonmember) # There are now three nonmembers. self.assertEqual( {address.email for address in self._mlist.nonmembers.addresses}, {'aperson@example.com', 'anne.person@example.com', 'a.person@example.com', }) # Let's now add one of Anne's addresses as a member. This deletes all # of Anne's nonmember memberships. anne_member = add_member( self._mlist, RequestRecord('a.person@example.com', 'Anne Person', DeliveryMode.regular, system_preferences.preferred_language), MemberRole.member) self.assertEqual(self._mlist.nonmembers.member_count, 0) members = list(self._mlist.members.members) self.assertEqual(len(members), 1) self.assertEqual(members[0], anne_member) class TestDeleteMember(unittest.TestCase): layer = ConfigLayer def setUp(self): self._mlist = create_list('test@example.com') def test_delete_member_not_a_member(self): # Try to delete an address which is not a member of the mailing list. with self.assertRaises(NotAMemberError) as cm: delete_member(self._mlist, 'noperson@example.com') self.assertEqual( str(cm.exception), 'noperson@example.com is not a member of test@example.com') mailman-3.2.2/src/mailman/app/tests/test_moderation.py0000644000175000017500000002052113442110351024145 0ustar maxkingmaxking00000000000000# Copyright (C) 2011-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Moderation tests.""" import unittest from mailman.app.lifecycle import create_list from mailman.app.moderator import ( handle_message, handle_unsubscription, hold_message, hold_unsubscription) from mailman.interfaces.action import Action from mailman.interfaces.member import MemberRole from mailman.interfaces.messages import IMessageStore from mailman.interfaces.requests import IListRequests from mailman.interfaces.subscriptions import ISubscriptionManager from mailman.interfaces.usermanager import IUserManager from mailman.runners.incoming import IncomingRunner from mailman.runners.outgoing import OutgoingRunner from mailman.runners.pipeline import PipelineRunner from mailman.testing.helpers import ( get_queue_messages, make_testable_runner, set_preferred, specialized_message_from_string as mfs) from mailman.testing.layers import SMTPLayer from mailman.utilities.datetime import now from zope.component import getUtility class TestModeration(unittest.TestCase): """Test moderation functionality.""" layer = SMTPLayer def setUp(self): self._mlist = create_list('test@example.com') self._request_db = IListRequests(self._mlist) self._msg = mfs("""\ From: anne@example.com To: test@example.com Subject: hold me Message-ID: """) self._in = make_testable_runner(IncomingRunner, 'in') self._pipeline = make_testable_runner(PipelineRunner, 'pipeline') self._out = make_testable_runner(OutgoingRunner, 'out') def test_accepted_message_gets_posted(self): # A message that is accepted by the moderator should get posted to the # mailing list. LP: #827697 msgdata = dict(listname='test@example.com', recipients=['bart@example.com']) request_id = hold_message(self._mlist, self._msg, msgdata) handle_message(self._mlist, request_id, Action.accept) self._in.run() self._pipeline.run() self._out.run() messages = list(SMTPLayer.smtpd.messages) self.assertEqual(len(messages), 1) message = messages[0] # We don't need to test the entire posted message, just the bits that # prove it got sent out. self.assertIn('x-mailman-version', message) self.assertIn('x-peer', message) # The X-Mailman-Approved-At header has local timezone information in # it, so test that separately. self.assertEqual(message['x-mailman-approved-at'][:-5], 'Mon, 01 Aug 2005 07:49:23 ') del message['x-mailman-approved-at'] # The Message-ID matches the original. self.assertEqual(message['message-id'], '') # Anne sent the message and the mailing list received it. self.assertEqual(message['from'], 'anne@example.com') self.assertEqual(message['to'], 'test@example.com') # The Subject header has the list's prefix. self.assertEqual(message['subject'], '[Test] hold me') # The list's -bounce address is the actual sender, and Bart is the # only actual recipient. These headers are added by the testing # framework and don't show up in production. They match the RFC 5321 # envelope. self.assertEqual(message['x-mailfrom'], 'test-bounces@example.com') self.assertEqual(message['x-rcptto'], 'bart@example.com') def test_hold_action_alias_for_defer(self): # In handle_message(), the 'hold' action is the same as 'defer' for # purposes of this API. request_id = hold_message(self._mlist, self._msg) handle_message(self._mlist, request_id, Action.defer) # The message is still in the pending requests. key, data = self._request_db.get_request(request_id) self.assertEqual(key, '') handle_message(self._mlist, request_id, Action.hold) key, data = self._request_db.get_request(request_id) self.assertEqual(key, '') def test_lp_1031391(self): # LP: #1031391 msgdata['received_time'] gets added by the LMTP server. # The value is a datetime. If this message gets held, it will break # pending requests since they require string keys and values. received_time = now() msgdata = dict(received_time=received_time) request_id = hold_message(self._mlist, self._msg, msgdata) key, data = self._request_db.get_request(request_id) self.assertEqual(data['received_time'], received_time) def test_forward(self): # We can forward the message to an email address. request_id = hold_message(self._mlist, self._msg) handle_message(self._mlist, request_id, Action.discard, forward=['zack@example.com']) # The forwarded message lives in the virgin queue. items = get_queue_messages('virgin', expected_count=1) self.assertEqual(str(items[0].msg['subject']), 'Forward of moderated message') self.assertEqual(items[0].msgdata['recipients'], ['zack@example.com']) def test_survive_a_deleted_message(self): # When the message that should be deleted is not found in the store, # no error is raised. request_id = hold_message(self._mlist, self._msg) message_store = getUtility(IMessageStore) message_store.delete_message('') handle_message(self._mlist, request_id, Action.discard) self.assertEqual(self._request_db.count, 0) def test_handled_message_stays_in_store(self): # The message is still available in the store, even when it's been # disposed of. request_id = hold_message(self._mlist, self._msg) handle_message(self._mlist, request_id, Action.discard) self.assertEqual(self._request_db.count, 0) message = getUtility(IMessageStore).get_message_by_id('') self.assertEqual(message['subject'], 'hold me') class TestUnsubscription(unittest.TestCase): """Test unsubscription requests.""" layer = SMTPLayer def setUp(self): self._mlist = create_list('test@example.com') self._manager = ISubscriptionManager(self._mlist) def test_unsubscribe_defer(self): # When unsubscriptions must be approved by the moderator, but the # moderator defers this decision. user_manager = getUtility(IUserManager) anne = user_manager.create_address('anne@example.org', 'Anne Person') token, token_owner, member = self._manager.register( anne, pre_verified=True, pre_confirmed=True, pre_approved=True) self.assertIsNone(token) self.assertEqual(member.address.email, 'anne@example.org') bart = user_manager.create_user('bart@example.com', 'Bart User') address = set_preferred(bart) self._mlist.subscribe(address, MemberRole.moderator) # Now hold and handle an unsubscription request. token = hold_unsubscription(self._mlist, 'anne@example.org') handle_unsubscription(self._mlist, token, Action.defer) items = get_queue_messages('virgin', expected_count=2) # Find the moderator message. for item in items: if item.msg['to'] == 'test-owner@example.com': break else: raise AssertionError('No moderator email found') self.assertEqual( item.msgdata['recipients'], {'test-owner@example.com'}) self.assertEqual( item.msg['subject'], 'New unsubscription request from Test by anne@example.org') def test_bogus_token(self): # Try to handle an unsubscription with a bogus token. self.assertRaises(LookupError, self._manager.confirm, None) mailman-3.2.2/src/mailman/app/tests/test_notifications.py0000644000175000017500000003376513442110351024673 0ustar maxkingmaxking00000000000000 # Copyright (C) 2012-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Test notifications.""" import os import unittest from contextlib import ExitStack from mailman.app.lifecycle import create_list from mailman.config import config from mailman.interfaces.languages import ILanguageManager from mailman.interfaces.member import MemberRole from mailman.interfaces.subscriptions import ISubscriptionManager from mailman.interfaces.template import ITemplateManager from mailman.interfaces.usermanager import IUserManager from mailman.testing.helpers import ( get_queue_messages, set_preferred, subscribe) from mailman.testing.layers import ConfigLayer from mailman.utilities.datetime import now from tempfile import TemporaryDirectory from zope.component import getUtility class TestNotifications(unittest.TestCase): """Test notifications.""" layer = ConfigLayer maxDiff = None def setUp(self): resources = ExitStack() self.addCleanup(resources.close) self.var_dir = resources.enter_context(TemporaryDirectory()) self._mlist = create_list('test@example.com') self._mlist.display_name = 'Test List' getUtility(ITemplateManager).set( 'list:user:notice:welcome', self._mlist.list_id, 'mailman:///welcome.txt') config.push('template config', """\ [paths.testing] template_dir: {}/templates """.format(self.var_dir)) resources.callback(config.pop, 'template config') # Populate the template directories with a few fake templates. path = os.path.join(self.var_dir, 'templates', 'site', 'en') os.makedirs(path) full_path = os.path.join(path, 'list:user:notice:welcome.txt') with open(full_path, 'w', encoding='utf-8') as fp: print("""\ Welcome to the $list_name mailing list. Posting address: $fqdn_listname Help and other requests: $list_requests Your name: $user_name Your address: $user_address""", file=fp) # Write a list-specific welcome message. path = os.path.join(self.var_dir, 'templates', 'lists', 'test@example.com', 'xx') os.makedirs(path) full_path = os.path.join(path, 'list:user:notice:welcome.txt') with open(full_path, 'w', encoding='utf-8') as fp: print('You just joined the $list_name mailing list!', file=fp) # Write a list-specific welcome message with non-ascii. path = os.path.join(self.var_dir, 'templates', 'lists', 'test@example.com', 'yy') os.makedirs(path) full_path = os.path.join(path, 'list:user:notice:welcome.txt') with open(full_path, 'w', encoding='utf-8') as fp: print('Yöu just joined the $list_name mailing list!', file=fp) # Write a list-specific address confirmation message with non-ascii. full_path = os.path.join(path, 'list:user:action:subscribe.txt') with open(full_path, 'w', encoding='utf-8') as fp: print('Wé need your confirmation', file=fp) def test_welcome_message(self): subscribe(self._mlist, 'Anne', email='anne@example.com') # Now there's one message in the virgin queue. items = get_queue_messages('virgin', expected_count=1) message = items[0].msg self.assertEqual(str(message['subject']), 'Welcome to the "Test List" mailing list') self.assertMultiLineEqual(message.get_payload(), """\ Welcome to the Test List mailing list. Posting address: test@example.com Help and other requests: test-request@example.com Your name: Anne Person Your address: anne@example.com """) def test_more_specific_welcome_message_nonenglish(self): # The welcome message url can contain placeholders for the fqdn list # name and language. getUtility(ITemplateManager).set( 'list:user:notice:welcome', self._mlist.list_id, 'mailman:///$listname/$language/welcome.txt') # Add the xx language and subscribe Anne using it. manager = getUtility(ILanguageManager) manager.add('xx', 'us-ascii', 'Xlandia') # We can't use the subscribe() helper because that would send the # welcome message before we set the member's preferred language. address = getUtility(IUserManager).create_address( 'anne@example.com', 'Anne Person') address.preferences.preferred_language = 'xx' self._mlist.subscribe(address) # Now there's one message in the virgin queue. items = get_queue_messages('virgin', expected_count=1) message = items[0].msg self.assertEqual(str(message['subject']), 'Welcome to the "Test List" mailing list') self.assertMultiLineEqual( message.get_payload(), 'You just joined the Test List mailing list!') def test_more_specific_messages_nonascii(self): # The welcome message url can contain placeholders for the fqdn list # name and language. getUtility(ITemplateManager).set( 'list:user:notice:welcome', self._mlist.list_id, 'mailman:///$listname/$language/welcome.txt') # Add the yy language and subscribe Anne using it. getUtility(ILanguageManager).add('yy', 'utf-8', 'Ylandia') # We can't use the subscribe() helper because that would send the # welcome message before we set the member's preferred language. address = getUtility(IUserManager).create_address( 'anne@example.com', 'Anné Person') address.preferences.preferred_language = 'yy' # Get the admin notice too. self._mlist.admin_notify_mchanges = True # Make another non-ascii replacement. self._mlist.display_name = 'Tést List' # And set the list's language. self._mlist.preferred_language = 'yy' self._mlist.subscribe(address) # Now there are two messages in the virgin queue. items = get_queue_messages('virgin', expected_count=2) if str(items[0].msg['subject']).startswith('Welcome'): welcome = items[0].msg admin_notice = items[1].msg else: welcome = items[1].msg admin_notice = items[0].msg self.assertEqual(str(welcome['subject']), 'Welcome to the "Tést List" mailing list') self.assertMultiLineEqual( welcome.get_payload(decode=True).decode('utf-8'), 'Yöu just joined the Tést List mailing list!') # Ensure the message is single part and properly encoded. raw_payload = welcome.get_payload() self.assertEqual( raw_payload.encode('us-ascii', 'replace').decode('us-ascii'), raw_payload) self.assertEqual(str(admin_notice['subject']), 'Tést List subscription notification') self.assertMultiLineEqual( admin_notice.get_payload(decode=True).decode('utf-8'), '=?utf-8?q?Ann=C3=A9_Person?= has been' ' successfully subscribed to Tést List.\n') # Ensure the message is single part and properly encoded. raw_payload = admin_notice.get_payload() self.assertEqual( raw_payload.encode('us-ascii', 'replace').decode('us-ascii'), raw_payload) def test_confirmation_message(self): # Create an address to subscribe. address = getUtility(IUserManager).create_address( 'anne@example.com', 'Anne Person') # Register the address with the list to create a confirmation notice. ISubscriptionManager(self._mlist).register(address) # Now there's one message in the virgin queue. items = get_queue_messages('virgin', expected_count=1) message = items[0].msg self.assertTrue(str(message['subject']).startswith('confirm')) self.assertMultiLineEqual( message.get_payload(), """\ Email Address Registration Confirmation Hello, this is the GNU Mailman server at example.com. We have received a registration request for the email address anne@example.com Before you can start using GNU Mailman at this site, you must first confirm that this is your email address. You can do this by replying to this message, keeping the Subject header intact. If you do not wish to register this email address, simply disregard this message. If you think you are being maliciously subscribed to the list, or have any other questions, you may contact test-owner@example.com """) def test_nonascii_confirmation_message(self): # Add the 'yy' language and set it getUtility(ILanguageManager).add('yy', 'utf-8', 'Ylandia') self._mlist.preferred_language = 'yy' # Create an address to subscribe. address = getUtility(IUserManager).create_address( 'anne@example.com', 'Anne Person') # Register the address with the list to create a confirmation notice. ISubscriptionManager(self._mlist).register(address) # Now there's one message in the virgin queue. items = get_queue_messages('virgin', expected_count=1) message = items[0].msg self.assertTrue(str(message['subject']).startswith('confirm')) self.assertMultiLineEqual( message.get_payload(decode=True).decode('utf-8'), 'Wé need your confirmation\n') def test_no_welcome_message_to_owners(self): # Welcome messages go only to mailing list members, not to owners. subscribe(self._mlist, 'Anne', MemberRole.owner, 'anne@example.com') # There is no welcome message in the virgin queue. get_queue_messages('virgin', expected_count=0) def test_no_welcome_message_to_nonmembers(self): # Welcome messages go only to mailing list members, not to nonmembers. subscribe(self._mlist, 'Anne', MemberRole.nonmember, 'anne@example.com') # There is no welcome message in the virgin queue. get_queue_messages('virgin', expected_count=0) def test_no_welcome_message_to_moderators(self): # Welcome messages go only to mailing list members, not to moderators. subscribe(self._mlist, 'Anne', MemberRole.moderator, 'anne@example.com') # There is no welcome message in the virgin queue. get_queue_messages('virgin', expected_count=0) def test_member_susbcribed_address_has_display_name(self): address = getUtility(IUserManager).create_address( 'anne@example.com', 'Anne Person') address.verified_on = now() self._mlist.subscribe(address) items = get_queue_messages('virgin', expected_count=1) message = items[0].msg self.assertEqual(message['to'], 'Anne Person ') def test_member_subscribed_address_has_no_display_name(self): address = getUtility(IUserManager).create_address('anne@example.com') address.verified_on = now() self._mlist.subscribe(address) items = get_queue_messages('virgin', expected_count=1) message = items[0].msg self.assertEqual(message['to'], 'anne@example.com') def test_member_is_user_and_has_display_name(self): user = getUtility(IUserManager).create_user( 'anne@example.com', 'Anne Person') set_preferred(user) self._mlist.subscribe(user) items = get_queue_messages('virgin', expected_count=1) message = items[0].msg self.assertEqual(message['to'], 'Anne Person ') def test_member_is_user_and_has_no_display_name(self): user = getUtility(IUserManager).create_user('anne@example.com') set_preferred(user) self._mlist.subscribe(user) items = get_queue_messages('virgin', expected_count=1) message = items[0].msg self.assertEqual(message['to'], 'anne@example.com') def test_member_has_linked_user_display_name(self): user = getUtility(IUserManager).create_user( 'anne@example.com', 'Anne Person') set_preferred(user) address = getUtility(IUserManager).create_address('anne2@example.com') address.verified_on = now() user.link(address) self._mlist.subscribe(address) items = get_queue_messages('virgin', expected_count=1) message = items[0].msg self.assertEqual(message['to'], 'Anne Person ') def test_member_has_no_linked_display_name(self): user = getUtility(IUserManager).create_user('anne@example.com') set_preferred(user) address = getUtility(IUserManager).create_address('anne2@example.com') address.verified_on = now() user.link(address) self._mlist.subscribe(address) items = get_queue_messages('virgin', expected_count=1) message = items[0].msg self.assertEqual(message['to'], 'anne2@example.com') def test_member_has_address_and_user_display_name(self): user = getUtility(IUserManager).create_user( 'anne@example.com', 'Anne Person') set_preferred(user) address = getUtility(IUserManager).create_address( 'anne2@example.com', 'Anne X Person') address.verified_on = now() user.link(address) self._mlist.subscribe(address) items = get_queue_messages('virgin', expected_count=1) message = items[0].msg self.assertEqual(message['to'], 'Anne X Person ') mailman-3.2.2/src/mailman/app/tests/test_subscriptions.py0000644000175000017500000010526613442110351024725 0ustar maxkingmaxking00000000000000# Copyright (C) 2011-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Tests for the subscription service.""" import unittest from contextlib import suppress from mailman.app.lifecycle import create_list from mailman.app.subscriptions import SubscriptionWorkflow from mailman.interfaces.bans import IBanManager from mailman.interfaces.mailinglist import SubscriptionPolicy from mailman.interfaces.member import MemberRole, MembershipIsBannedError from mailman.interfaces.pending import IPendings from mailman.interfaces.subscriptions import TokenOwner from mailman.interfaces.usermanager import IUserManager from mailman.testing.helpers import ( LogFileMark, get_queue_messages, set_preferred) from mailman.testing.layers import ConfigLayer from mailman.utilities.datetime import now from unittest.mock import patch from zope.component import getUtility class TestSubscriptionWorkflow(unittest.TestCase): layer = ConfigLayer maxDiff = None def setUp(self): self._mlist = create_list('test@example.com') self._mlist.admin_immed_notify = False self._anne = 'anne@example.com' self._user_manager = getUtility(IUserManager) self._expected_pendings_count = 0 def tearDown(self): # There usually should be no pending after all is said and done, but # some tests don't complete the workflow. self.assertEqual(getUtility(IPendings).count, self._expected_pendings_count) def test_start_state(self): # The workflow starts with no tokens or member. workflow = SubscriptionWorkflow(self._mlist) self.assertIsNone(workflow.token) self.assertEqual(workflow.token_owner, TokenOwner.no_one) self.assertIsNone(workflow.member) def test_pended_data(self): # There is a Pendable associated with the held request, and it has # some data associated with it. anne = self._user_manager.create_address(self._anne) workflow = SubscriptionWorkflow(self._mlist, anne) with suppress(StopIteration): workflow.run_thru('send_confirmation') self.assertIsNotNone(workflow.token) pendable = getUtility(IPendings).confirm(workflow.token, expunge=False) self.assertEqual(pendable['list_id'], 'test.example.com') self.assertEqual(pendable['email'], 'anne@example.com') self.assertEqual(pendable['display_name'], '') self.assertEqual(pendable['when'], '2005-08-01T07:49:23') self.assertEqual(pendable['token_owner'], 'subscriber') # The token is still in the database. self._expected_pendings_count = 1 def test_user_or_address_required(self): # The `subscriber` attribute must be a user or address. workflow = SubscriptionWorkflow(self._mlist) self.assertRaises(AssertionError, list, workflow) def test_sanity_checks_address(self): # Ensure that the sanity check phase, when given an IAddress, ends up # with a linked user. anne = self._user_manager.create_address(self._anne) workflow = SubscriptionWorkflow(self._mlist, anne) self.assertIsNotNone(workflow.address) self.assertIsNone(workflow.user) workflow.run_thru('sanity_checks') self.assertIsNotNone(workflow.address) self.assertIsNotNone(workflow.user) self.assertEqual(list(workflow.user.addresses)[0].email, self._anne) def test_sanity_checks_user_with_preferred_address(self): # Ensure that the sanity check phase, when given an IUser with a # preferred address, ends up with an address. anne = self._user_manager.make_user(self._anne) address = set_preferred(anne) workflow = SubscriptionWorkflow(self._mlist, anne) # The constructor sets workflow.address because the user has a # preferred address. self.assertEqual(workflow.address, address) self.assertEqual(workflow.user, anne) workflow.run_thru('sanity_checks') self.assertEqual(workflow.address, address) self.assertEqual(workflow.user, anne) def test_sanity_checks_user_without_preferred_address(self): # Ensure that the sanity check phase, when given a user without a # preferred address, but with at least one linked address, gets an # address. anne = self._user_manager.make_user(self._anne) workflow = SubscriptionWorkflow(self._mlist, anne) self.assertIsNone(workflow.address) self.assertEqual(workflow.user, anne) workflow.run_thru('sanity_checks') self.assertIsNotNone(workflow.address) self.assertEqual(workflow.user, anne) def test_sanity_checks_user_with_multiple_linked_addresses(self): # Ensure that the santiy check phase, when given a user without a # preferred address, but with multiple linked addresses, gets of of # those addresses (exactly which one is undefined). anne = self._user_manager.make_user(self._anne) anne.link(self._user_manager.create_address('anne@example.net')) anne.link(self._user_manager.create_address('anne@example.org')) workflow = SubscriptionWorkflow(self._mlist, anne) self.assertIsNone(workflow.address) self.assertEqual(workflow.user, anne) workflow.run_thru('sanity_checks') self.assertIn(workflow.address.email, ['anne@example.com', 'anne@example.net', 'anne@example.org']) self.assertEqual(workflow.user, anne) def test_sanity_checks_user_without_addresses(self): # It is an error to try to subscribe a user with no linked addresses. user = self._user_manager.create_user() workflow = SubscriptionWorkflow(self._mlist, user) self.assertRaises(AssertionError, workflow.run_thru, 'sanity_checks') def test_sanity_checks_globally_banned_address(self): # An exception is raised if the address is globally banned. anne = self._user_manager.create_address(self._anne) IBanManager(None).ban(self._anne) workflow = SubscriptionWorkflow(self._mlist, anne) self.assertRaises(MembershipIsBannedError, list, workflow) def test_sanity_checks_banned_address(self): # An exception is raised if the address is banned by the mailing list. anne = self._user_manager.create_address(self._anne) IBanManager(self._mlist).ban(self._anne) workflow = SubscriptionWorkflow(self._mlist, anne) self.assertRaises(MembershipIsBannedError, list, workflow) def test_verification_checks_with_verified_address(self): # When the address is already verified, we skip straight to the # confirmation checks. anne = self._user_manager.create_address(self._anne) anne.verified_on = now() workflow = SubscriptionWorkflow(self._mlist, anne) workflow.run_thru('verification_checks') with patch.object(workflow, '_step_confirmation_checks') as step: next(workflow) step.assert_called_once_with() def test_verification_checks_with_pre_verified_address(self): # When the address is not yet verified, but the pre-verified flag is # passed to the workflow, we skip to the confirmation checks. anne = self._user_manager.create_address(self._anne) workflow = SubscriptionWorkflow(self._mlist, anne, pre_verified=True) workflow.run_thru('verification_checks') with patch.object(workflow, '_step_confirmation_checks') as step: next(workflow) step.assert_called_once_with() # And now the address is verified. self.assertIsNotNone(anne.verified_on) def test_verification_checks_confirmation_needed(self): # The address is neither verified, nor is the pre-verified flag set. # A confirmation message must be sent to the user which will also # verify their address. anne = self._user_manager.create_address(self._anne) workflow = SubscriptionWorkflow(self._mlist, anne) workflow.run_thru('verification_checks') with patch.object(workflow, '_step_send_confirmation') as step: next(workflow) step.assert_called_once_with() # The address still hasn't been verified. self.assertIsNone(anne.verified_on) def test_confirmation_checks_open_list(self): # A subscription to an open list does not need to be confirmed or # moderated. self._mlist.subscription_policy = SubscriptionPolicy.open anne = self._user_manager.create_address(self._anne) workflow = SubscriptionWorkflow(self._mlist, anne, pre_verified=True) workflow.run_thru('confirmation_checks') with patch.object(workflow, '_step_do_subscription') as step: next(workflow) step.assert_called_once_with() def test_confirmation_checks_no_user_confirmation_needed(self): # A subscription to a list which does not need user confirmation skips # to the moderation checks. self._mlist.subscription_policy = SubscriptionPolicy.moderate anne = self._user_manager.create_address(self._anne) workflow = SubscriptionWorkflow(self._mlist, anne, pre_verified=True) workflow.run_thru('confirmation_checks') with patch.object(workflow, '_step_moderation_checks') as step: next(workflow) step.assert_called_once_with() def test_confirmation_checks_confirm_pre_confirmed(self): # The subscription policy requires user confirmation, but their # subscription is pre-confirmed. Since moderation is not required, # the user will be immediately subscribed. self._mlist.subscription_policy = SubscriptionPolicy.confirm anne = self._user_manager.create_address(self._anne) workflow = SubscriptionWorkflow(self._mlist, anne, pre_verified=True, pre_confirmed=True) workflow.run_thru('confirmation_checks') with patch.object(workflow, '_step_do_subscription') as step: next(workflow) step.assert_called_once_with() def test_confirmation_checks_confirm_then_moderate_pre_confirmed(self): # The subscription policy requires user confirmation, but their # subscription is pre-confirmed. Since moderation is required, that # check will be performed. self._mlist.subscription_policy = ( SubscriptionPolicy.confirm_then_moderate) anne = self._user_manager.create_address(self._anne) workflow = SubscriptionWorkflow(self._mlist, anne, pre_verified=True, pre_confirmed=True) workflow.run_thru('confirmation_checks') with patch.object(workflow, '_step_moderation_checks') as step: next(workflow) step.assert_called_once_with() def test_confirmation_checks_confirm_and_moderate_pre_confirmed(self): # The subscription policy requires user confirmation and moderation, # but their subscription is pre-confirmed. self._mlist.subscription_policy = ( SubscriptionPolicy.confirm_then_moderate) anne = self._user_manager.create_address(self._anne) workflow = SubscriptionWorkflow(self._mlist, anne, pre_verified=True, pre_confirmed=True) workflow.run_thru('confirmation_checks') with patch.object(workflow, '_step_moderation_checks') as step: next(workflow) step.assert_called_once_with() def test_confirmation_checks_confirmation_needed(self): # The subscription policy requires confirmation and the subscription # is not pre-confirmed. self._mlist.subscription_policy = SubscriptionPolicy.confirm anne = self._user_manager.create_address(self._anne) workflow = SubscriptionWorkflow(self._mlist, anne, pre_verified=True) workflow.run_thru('confirmation_checks') with patch.object(workflow, '_step_send_confirmation') as step: next(workflow) step.assert_called_once_with() def test_confirmation_checks_moderate_confirmation_needed(self): # The subscription policy requires confirmation and moderation, and the # subscription is not pre-confirmed. self._mlist.subscription_policy = ( SubscriptionPolicy.confirm_then_moderate) anne = self._user_manager.create_address(self._anne) workflow = SubscriptionWorkflow(self._mlist, anne, pre_verified=True) workflow.run_thru('confirmation_checks') with patch.object(workflow, '_step_send_confirmation') as step: next(workflow) step.assert_called_once_with() def test_moderation_checks_pre_approved(self): # The subscription is pre-approved by the moderator. self._mlist.subscription_policy = SubscriptionPolicy.moderate anne = self._user_manager.create_address(self._anne) workflow = SubscriptionWorkflow(self._mlist, anne, pre_verified=True, pre_approved=True) workflow.run_thru('moderation_checks') with patch.object(workflow, '_step_do_subscription') as step: next(workflow) step.assert_called_once_with() def test_moderation_checks_approval_required(self): # The moderator must approve the subscription. self._mlist.subscription_policy = SubscriptionPolicy.moderate anne = self._user_manager.create_address(self._anne) workflow = SubscriptionWorkflow(self._mlist, anne, pre_verified=True) workflow.run_thru('moderation_checks') with patch.object(workflow, '_step_get_moderator_approval') as step: next(workflow) step.assert_called_once_with() def test_do_subscription(self): # An open subscription policy plus a pre-verified address means the # user gets subscribed to the mailing list without any further # confirmations or approvals. self._mlist.subscription_policy = SubscriptionPolicy.open anne = self._user_manager.create_address(self._anne) workflow = SubscriptionWorkflow(self._mlist, anne, pre_verified=True) # Consume the entire state machine. list(workflow) # Anne is now a member of the mailing list. member = self._mlist.regular_members.get_member(self._anne) self.assertEqual(member.address, anne) self.assertEqual(workflow.member, member) # No further token is needed. self.assertIsNone(workflow.token) self.assertEqual(workflow.token_owner, TokenOwner.no_one) def test_do_subscription_pre_approved(self): # An moderation-requiring subscription policy plus a pre-verified and # pre-approved address means the user gets subscribed to the mailing # list without any further confirmations or approvals. self._mlist.subscription_policy = SubscriptionPolicy.moderate anne = self._user_manager.create_address(self._anne) workflow = SubscriptionWorkflow(self._mlist, anne, pre_verified=True, pre_approved=True) # Consume the entire state machine. list(workflow) # Anne is now a member of the mailing list. member = self._mlist.regular_members.get_member(self._anne) self.assertEqual(member.address, anne) self.assertEqual(workflow.member, member) # No further token is needed. self.assertIsNone(workflow.token) self.assertEqual(workflow.token_owner, TokenOwner.no_one) def test_do_subscription_pre_approved_pre_confirmed(self): # An moderation-requiring subscription policy plus a pre-verified and # pre-approved address means the user gets subscribed to the mailing # list without any further confirmations or approvals. self._mlist.subscription_policy = ( SubscriptionPolicy.confirm_then_moderate) anne = self._user_manager.create_address(self._anne) workflow = SubscriptionWorkflow(self._mlist, anne, pre_verified=True, pre_confirmed=True, pre_approved=True) # Consume the entire state machine. list(workflow) # Anne is now a member of the mailing list. member = self._mlist.regular_members.get_member(self._anne) self.assertEqual(member.address, anne) self.assertEqual(workflow.member, member) # No further token is needed. self.assertIsNone(workflow.token) self.assertEqual(workflow.token_owner, TokenOwner.no_one) def test_do_subscription_cleanups(self): # Once the user is subscribed, the token, and its associated pending # database record will be removed from the database. self._mlist.subscription_policy = SubscriptionPolicy.open anne = self._user_manager.create_address(self._anne) workflow = SubscriptionWorkflow(self._mlist, anne, pre_verified=True, pre_confirmed=True, pre_approved=True) # Consume the entire state machine. list(workflow) # Anne is now a member of the mailing list. member = self._mlist.regular_members.get_member(self._anne) self.assertEqual(member.address, anne) self.assertEqual(workflow.member, member) # The workflow is done, so it has no token. self.assertIsNone(workflow.token) self.assertEqual(workflow.token_owner, TokenOwner.no_one) def test_moderator_approves(self): # The workflow runs until moderator approval is required, at which # point the workflow is saved. Once the moderator approves, the # workflow resumes and the user is subscribed. self._mlist.subscription_policy = SubscriptionPolicy.moderate anne = self._user_manager.create_address(self._anne) workflow = SubscriptionWorkflow(self._mlist, anne, pre_verified=True, pre_confirmed=True) # Consume the entire state machine. list(workflow) # The user is not currently subscribed to the mailing list. member = self._mlist.regular_members.get_member(self._anne) self.assertIsNone(member) self.assertIsNone(workflow.member) # The token is owned by the moderator. self.assertIsNotNone(workflow.token) self.assertEqual(workflow.token_owner, TokenOwner.moderator) # Create a new workflow with the previous workflow's save token, and # restore its state. This models an approved subscription and should # result in the user getting subscribed. approved_workflow = SubscriptionWorkflow(self._mlist) approved_workflow.token = workflow.token approved_workflow.restore() list(approved_workflow) # Now the user is subscribed to the mailing list. member = self._mlist.regular_members.get_member(self._anne) self.assertEqual(member.address, anne) self.assertEqual(approved_workflow.member, member) # No further token is needed. self.assertIsNone(approved_workflow.token) self.assertEqual(approved_workflow.token_owner, TokenOwner.no_one) def test_get_moderator_approval_log_on_hold(self): # When the subscription is held for moderator approval, a message is # logged. mark = LogFileMark('mailman.subscribe') self._mlist.subscription_policy = SubscriptionPolicy.moderate anne = self._user_manager.create_address(self._anne) workflow = SubscriptionWorkflow(self._mlist, anne, pre_verified=True, pre_confirmed=True) # Consume the entire state machine. list(workflow) self.assertIn( 'test@example.com: held subscription request from anne@example.com', mark.readline() ) # The state machine stopped at the moderator approval so there will be # one token still in the database. self._expected_pendings_count = 1 def test_get_moderator_approval_notifies_moderators(self): # When the subscription is held for moderator approval, and the list # is so configured, a notification is sent to the list moderators. self._mlist.admin_immed_notify = True self._mlist.subscription_policy = SubscriptionPolicy.moderate anne = self._user_manager.create_address(self._anne) bart = self._user_manager.create_user('bart@example.com', 'Bart User') address = set_preferred(bart) self._mlist.subscribe(address, MemberRole.moderator) workflow = SubscriptionWorkflow(self._mlist, anne, pre_verified=True, pre_confirmed=True) # Consume the entire state machine. list(workflow) # Find the moderator message. items = get_queue_messages('virgin', expected_count=1) for item in items: if item.msg['to'] == 'test-owner@example.com': break else: raise AssertionError('No moderator email found') self.assertEqual( item.msgdata['recipients'], {'test-owner@example.com'}) message = items[0].msg self.assertEqual(message['From'], 'test-owner@example.com') self.assertEqual(message['To'], 'test-owner@example.com') self.assertEqual( message['Subject'], 'New subscription request to Test from anne@example.com') self.assertEqual(message.get_payload(), """\ Your authorization is required for a mailing list subscription request approval: For: anne@example.com List: test@example.com """) # The state machine stopped at the moderator approval so there will be # one token still in the database. self._expected_pendings_count = 1 def test_get_moderator_approval_no_notifications(self): # When the subscription is held for moderator approval, and the list # is so configured, a notification is sent to the list moderators. self._mlist.admin_immed_notify = False self._mlist.subscription_policy = SubscriptionPolicy.moderate anne = self._user_manager.create_address(self._anne) workflow = SubscriptionWorkflow(self._mlist, anne, pre_verified=True, pre_confirmed=True) # Consume the entire state machine. list(workflow) get_queue_messages('virgin', expected_count=0) # The state machine stopped at the moderator approval so there will be # one token still in the database. self._expected_pendings_count = 1 def test_send_confirmation(self): # A confirmation message gets sent when the address is not verified. anne = self._user_manager.create_address(self._anne) self.assertIsNone(anne.verified_on) # Run the workflow to model the confirmation step. workflow = SubscriptionWorkflow(self._mlist, anne) list(workflow) items = get_queue_messages('virgin', expected_count=1) message = items[0].msg token = workflow.token self.assertEqual(message['Subject'], 'confirm {}'.format(token)) self.assertEqual( message['From'], 'test-confirm+{}@example.com'.format(token)) # The confirmation message is not `Precedence: bulk`. self.assertIsNone(message['precedence']) # The state machine stopped at the moderator approval so there will be # one token still in the database. self._expected_pendings_count = 1 def test_send_confirmation_pre_confirmed(self): # A confirmation message gets sent when the address is not verified # but the subscription is pre-confirmed. anne = self._user_manager.create_address(self._anne) self.assertIsNone(anne.verified_on) # Run the workflow to model the confirmation step. workflow = SubscriptionWorkflow(self._mlist, anne, pre_confirmed=True) list(workflow) items = get_queue_messages('virgin', expected_count=1) message = items[0].msg token = workflow.token self.assertEqual( message['Subject'], 'confirm {}'.format(workflow.token)) self.assertEqual( message['From'], 'test-confirm+{}@example.com'.format(token)) # The state machine stopped at the moderator approval so there will be # one token still in the database. self._expected_pendings_count = 1 def test_send_confirmation_pre_verified(self): # A confirmation message gets sent even when the address is verified # when the subscription must be confirmed. self._mlist.subscription_policy = SubscriptionPolicy.confirm anne = self._user_manager.create_address(self._anne) self.assertIsNone(anne.verified_on) # Run the workflow to model the confirmation step. workflow = SubscriptionWorkflow(self._mlist, anne, pre_verified=True) list(workflow) items = get_queue_messages('virgin', expected_count=1) message = items[0].msg token = workflow.token self.assertEqual( message['Subject'], 'confirm {}'.format(workflow.token)) self.assertEqual( message['From'], 'test-confirm+{}@example.com'.format(token)) # The state machine stopped at the moderator approval so there will be # one token still in the database. self._expected_pendings_count = 1 def test_do_confirm_verify_address(self): # The address is not yet verified, nor are we pre-verifying. A # confirmation message will be sent. When the user confirms their # subscription request, the address will end up being verified. anne = self._user_manager.create_address(self._anne) self.assertIsNone(anne.verified_on) # Run the workflow to model the confirmation step. workflow = SubscriptionWorkflow(self._mlist, anne) list(workflow) # The address is still not verified. self.assertIsNone(anne.verified_on) confirm_workflow = SubscriptionWorkflow(self._mlist) confirm_workflow.token = workflow.token confirm_workflow.restore() confirm_workflow.run_thru('do_confirm_verify') # The address is now verified. self.assertIsNotNone(anne.verified_on) def test_do_confirm_verify_user(self): # A confirmation step is necessary when a user subscribes with their # preferred address, and we are not pre-confirming. anne = self._user_manager.create_user(self._anne) set_preferred(anne) # Run the workflow to model the confirmation step. There is no # subscriber attribute yet. workflow = SubscriptionWorkflow(self._mlist, anne) list(workflow) self.assertEqual(workflow.subscriber, anne) # Do a confirmation workflow, which should now set the subscriber. confirm_workflow = SubscriptionWorkflow(self._mlist) confirm_workflow.token = workflow.token confirm_workflow.restore() confirm_workflow.run_thru('do_confirm_verify') # The address is now verified. self.assertEqual(confirm_workflow.subscriber, anne) def test_do_confirmation_subscribes_user(self): # Subscriptions to the mailing list must be confirmed. Once that's # done, the user's address (which is not initially verified) gets # subscribed to the mailing list. self._mlist.subscription_policy = SubscriptionPolicy.confirm anne = self._user_manager.create_address(self._anne) self.assertIsNone(anne.verified_on) workflow = SubscriptionWorkflow(self._mlist, anne) list(workflow) # Anne is not yet a member. member = self._mlist.regular_members.get_member(self._anne) self.assertIsNone(member) self.assertIsNone(workflow.member) # The token is owned by the subscriber. self.assertIsNotNone(workflow.token) self.assertEqual(workflow.token_owner, TokenOwner.subscriber) # Confirm. confirm_workflow = SubscriptionWorkflow(self._mlist) confirm_workflow.token = workflow.token confirm_workflow.restore() list(confirm_workflow) self.assertIsNotNone(anne.verified_on) # Anne is now a member. member = self._mlist.regular_members.get_member(self._anne) self.assertEqual(member.address, anne) self.assertEqual(confirm_workflow.member, member) # No further token is needed. self.assertIsNone(confirm_workflow.token) self.assertEqual(confirm_workflow.token_owner, TokenOwner.no_one) def test_prevent_confirmation_replay_attacks(self): # Ensure that if the workflow requires two confirmations, e.g. first # the user confirming their subscription, and then the moderator # approving it, that different tokens are used in these two cases. self._mlist.subscription_policy = ( SubscriptionPolicy.confirm_then_moderate) anne = self._user_manager.create_address(self._anne) workflow = SubscriptionWorkflow(self._mlist, anne, pre_verified=True) # Run the state machine up to the first confirmation, and cache the # confirmation token. list(workflow) token = workflow.token # Anne is not yet a member of the mailing list. member = self._mlist.regular_members.get_member(self._anne) self.assertIsNone(member) self.assertIsNone(workflow.member) # The token is owned by the subscriber. self.assertIsNotNone(workflow.token) self.assertEqual(workflow.token_owner, TokenOwner.subscriber) # The old token will not work for moderator approval. moderator_workflow = SubscriptionWorkflow(self._mlist) moderator_workflow.token = token moderator_workflow.restore() list(moderator_workflow) # The token is owned by the moderator. self.assertIsNotNone(moderator_workflow.token) self.assertEqual(moderator_workflow.token_owner, TokenOwner.moderator) # While we wait for the moderator to approve the subscription, note # that there's a new token for the next steps. self.assertNotEqual(token, moderator_workflow.token) # The old token won't work. final_workflow = SubscriptionWorkflow(self._mlist) final_workflow.token = token self.assertRaises(LookupError, final_workflow.restore) # Running this workflow will fail. self.assertRaises(AssertionError, list, final_workflow) # Anne is still not subscribed. member = self._mlist.regular_members.get_member(self._anne) self.assertIsNone(member) self.assertIsNone(final_workflow.member) # However, if we use the new token, her subscription request will be # approved by the moderator. final_workflow.token = moderator_workflow.token final_workflow.restore() list(final_workflow) # And now Anne is a member. member = self._mlist.regular_members.get_member(self._anne) self.assertEqual(member.address.email, self._anne) self.assertEqual(final_workflow.member, member) # No further token is needed. self.assertIsNone(final_workflow.token) self.assertEqual(final_workflow.token_owner, TokenOwner.no_one) def test_confirmation_needed_and_pre_confirmed(self): # The subscription policy is 'confirm' but the subscription is # pre-confirmed so the moderation checks can be skipped. self._mlist.subscription_policy = SubscriptionPolicy.confirm anne = self._user_manager.create_address(self._anne) workflow = SubscriptionWorkflow( self._mlist, anne, pre_verified=True, pre_confirmed=True, pre_approved=True) list(workflow) # Anne was subscribed. self.assertIsNone(workflow.token) self.assertEqual(workflow.token_owner, TokenOwner.no_one) self.assertEqual(workflow.member.address, anne) def test_restore_user_absorbed(self): # The subscribing user is absorbed (and thus deleted) before the # moderator approves the subscription. self._mlist.subscription_policy = SubscriptionPolicy.moderate anne = self._user_manager.create_user(self._anne) bill = self._user_manager.create_user('bill@example.com') set_preferred(bill) # anne subscribes. workflow = SubscriptionWorkflow(self._mlist, anne, pre_verified=True) list(workflow) # bill absorbs anne. bill.absorb(anne) # anne's subscription request is approved. approved_workflow = SubscriptionWorkflow(self._mlist) approved_workflow.token = workflow.token approved_workflow.restore() self.assertEqual(approved_workflow.user, bill) # Run the workflow through. list(approved_workflow) def test_restore_address_absorbed(self): # The subscribing user is absorbed (and thus deleted) before the # moderator approves the subscription. self._mlist.subscription_policy = SubscriptionPolicy.moderate anne = self._user_manager.create_user(self._anne) anne_address = anne.addresses[0] bill = self._user_manager.create_user('bill@example.com') # anne subscribes. workflow = SubscriptionWorkflow( self._mlist, anne_address, pre_verified=True) list(workflow) # bill absorbs anne. bill.absorb(anne) self.assertIn(anne_address, bill.addresses) # anne's subscription request is approved. approved_workflow = SubscriptionWorkflow(self._mlist) approved_workflow.token = workflow.token approved_workflow.restore() self.assertEqual(approved_workflow.user, bill) # Run the workflow through. list(approved_workflow) mailman-3.2.2/src/mailman/app/tests/test_unsubscriptions.py0000644000175000017500000006067613442110351025275 0ustar maxkingmaxking00000000000000# Copyright (C) 2016-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Test for unsubscription service.""" import unittest from contextlib import suppress from mailman.app.lifecycle import create_list from mailman.app.subscriptions import UnSubscriptionWorkflow from mailman.interfaces.mailinglist import SubscriptionPolicy from mailman.interfaces.pending import IPendings from mailman.interfaces.subscriptions import TokenOwner from mailman.interfaces.usermanager import IUserManager from mailman.testing.helpers import LogFileMark, get_queue_messages from mailman.testing.layers import ConfigLayer from mailman.utilities.datetime import now from unittest.mock import patch from zope.component import getUtility class TestUnSubscriptionWorkflow(unittest.TestCase): layer = ConfigLayer maxDiff = None def setUp(self): self._mlist = create_list('test@example.com') self._mlist.admin_immed_notify = False self._mlist.unsubscription_policy = SubscriptionPolicy.open self._mlist.send_welcome_message = False self._anne = 'anne@example.com' self._user_manager = getUtility(IUserManager) self.anne = self._user_manager.create_user(self._anne) self.anne.addresses[0].verified_on = now() self.anne.preferred_address = self.anne.addresses[0] self._mlist.subscribe(self.anne) self._expected_pendings_count = 0 def tearDown(self): # There usually should be no pending after all is said and done, but # some tests don't complete the workflow. self.assertEqual(getUtility(IPendings).count, self._expected_pendings_count) def test_start_state(self): # Test the workflow starts with no tokens or members. workflow = UnSubscriptionWorkflow(self._mlist) self.assertEqual(workflow.token_owner, TokenOwner.no_one) self.assertIsNone(workflow.token) self.assertIsNone(workflow.member) def test_pended_data(self): # Test there is a Pendable object associated with a held # unsubscription request and it has some valid data associated with # it. self._mlist.unsubscription_policy = SubscriptionPolicy.confirm workflow = UnSubscriptionWorkflow(self._mlist, self.anne) with suppress(StopIteration): workflow.run_thru('send_confirmation') self.assertIsNotNone(workflow.token) pendable = getUtility(IPendings).confirm(workflow.token, expunge=False) self.assertEqual(pendable['list_id'], 'test.example.com') self.assertEqual(pendable['email'], 'anne@example.com') self.assertEqual(pendable['display_name'], '') self.assertEqual(pendable['when'], '2005-08-01T07:49:23') self.assertEqual(pendable['token_owner'], 'subscriber') # The token is still in the database. self._expected_pendings_count = 1 def test_user_or_address_required(self): # The `subscriber` attribute must be a user or address that is provided # to the workflow. workflow = UnSubscriptionWorkflow(self._mlist) self.assertRaises(AssertionError, list, workflow) def test_user_is_subscribed_to_unsubscribe(self): # A user must be subscribed to a list when trying to unsubscribe. addr = self._user_manager.create_address('aperson@example.org') addr.verfied_on = now() workflow = UnSubscriptionWorkflow(self._mlist, addr) self.assertRaises(AssertionError, workflow.run_thru, 'subscription_checks') def test_confirmation_checks_open_list(self): # An unsubscription from an open list does not need to be confirmed or # moderated. self._mlist.unsubscription_policy = SubscriptionPolicy.open workflow = UnSubscriptionWorkflow(self._mlist, self.anne) workflow.run_thru('confirmation_checks') with patch.object(workflow, '_step_do_unsubscription') as step: next(workflow) step.assert_called_once_with() def test_confirmation_checks_no_user_confirmation_needed(self): # An unsubscription from a list which does not need user confirmation # skips to the moderation checks. self._mlist.unsubscription_policy = SubscriptionPolicy.moderate workflow = UnSubscriptionWorkflow(self._mlist, self.anne, pre_confirmed=True) workflow.run_thru('confirmation_checks') with patch.object(workflow, '_step_moderation_checks') as step: next(workflow) step.assert_called_once_with() def test_confirmation_checks_confirm_pre_confirmed(self): # The unsubscription policy requires user-confirmation, but their # unsubscription is pre-confirmed. Since moderation is not reuqired, # the user will be immediately unsubscribed. self._mlist.unsubscription_policy = SubscriptionPolicy.confirm workflow = UnSubscriptionWorkflow( self._mlist, self.anne, pre_confirmed=True) workflow.run_thru('confirmation_checks') with patch.object(workflow, '_step_do_unsubscription') as step: next(workflow) step.assert_called_once_with() def test_confirmation_checks_confirm_then_moderate_pre_confirmed(self): # The unsubscription policy requires user confirmation, but their # unsubscription is pre-confirmed. Since moderation is required, that # check will be performed. self._mlist.unsubscription_policy = ( SubscriptionPolicy.confirm_then_moderate) workflow = UnSubscriptionWorkflow( self._mlist, self.anne, pre_confirmed=True) workflow.run_thru('confirmation_checks') with patch.object(workflow, '_step_do_unsubscription') as step: next(workflow) step.assert_called_once_with() def test_send_confirmation_checks_confirm_list(self): # The unsubscription policy requires user confirmation and the # unsubscription is not pre-confirmed. self._mlist.unsubscription_policy = SubscriptionPolicy.confirm workflow = UnSubscriptionWorkflow(self._mlist, self.anne) workflow.run_thru('confirmation_checks') with patch.object(workflow, '_step_send_confirmation') as step: next(workflow) step.assert_called_once_with() def test_moderation_checks_moderated_list(self): # The unsubscription policy requires moderation. self._mlist.unsubscription_policy = SubscriptionPolicy.moderate workflow = UnSubscriptionWorkflow(self._mlist, self.anne) workflow.run_thru('confirmation_checks') with patch.object(workflow, '_step_moderation_checks') as step: next(workflow) step.assert_called_once_with() def test_moderation_checks_approval_required(self): # The moderator must approve the subscription request. self._mlist.unsubscription_policy = SubscriptionPolicy.moderate workflow = UnSubscriptionWorkflow(self._mlist, self.anne) workflow.run_thru('moderation_checks') with patch.object(workflow, '_step_get_moderator_approval') as step: next(workflow) step.assert_called_once_with() def test_do_unsusbcription(self): # An open unsubscription policy means the user gets unsubscribed to # the mailing list without any further confirmations or approvals. self._mlist.unsubscription_policy = SubscriptionPolicy.open workflow = UnSubscriptionWorkflow(self._mlist, self.anne) list(workflow) member = self._mlist.regular_members.get_member(self._anne) self.assertIsNone(member) def test_do_unsubscription_pre_approved(self): # A moderation-requiring subscription policy plus a pre-approved # address means the user gets unsubscribed from the mailing list # without any further confirmation or approvals. self._mlist.unsubscription_policy = SubscriptionPolicy.moderate workflow = UnSubscriptionWorkflow(self._mlist, self.anne, pre_approved=True) list(workflow) # Anne is now unsubscribed form the mailing list. member = self._mlist.regular_members.get_member(self._anne) self.assertIsNone(member) # No further token is needed. self.assertIsNone(workflow.token) self.assertEqual(workflow.token_owner, TokenOwner.no_one) def test_do_unsubscription_pre_approved_pre_confirmed(self): # A moderation-requiring unsubscription policy plus a pre-appvoed # address means the user gets unsubscribed to the mailing list without # any further confirmations or approvals. self._mlist.unsubscription_policy = ( SubscriptionPolicy.confirm_then_moderate) workflow = UnSubscriptionWorkflow(self._mlist, self.anne, pre_approved=True, pre_confirmed=True) list(workflow) member = self._mlist.regular_members.get_member(self._anne) self.assertIsNone(member) # No further token is needed. self.assertIsNone(workflow.token) self.assertEqual(workflow.token_owner, TokenOwner.no_one) def test_do_unsubscription_cleanups(self): # Once the user is unsubscribed, the token and its associated pending # database record will be removed from the database. self._mlist.unsubscription_policy = SubscriptionPolicy.open workflow = UnSubscriptionWorkflow(self._mlist, self.anne, pre_approved=True, pre_confirmed=True) # Run the workflow. list(workflow) # Anne is now unsubscribed from the list. member = self._mlist.regular_members.get_member(self._anne) self.assertIsNone(member) # Workflow is done, so it has no token. self.assertIsNone(workflow.token) self.assertEqual(workflow.token_owner, TokenOwner.no_one) def test_moderator_approves(self): # The workflow runs until moderator approval is required, at which # point the workflow is saved. Once the moderator approves, the # workflow resumes and the user is unsubscribed. self._mlist.unsubscription_policy = SubscriptionPolicy.moderate workflow = UnSubscriptionWorkflow( self._mlist, self.anne, pre_confirmed=True) # Run the entire workflow. list(workflow) # The user is currently subscribed to the mailing list. member = self._mlist.regular_members.get_member(self._anne) self.assertIsNotNone(member) self.assertIsNotNone(workflow.member) # The token is owned by the moderator. self.assertIsNotNone(workflow.token) self.assertEqual(workflow.token_owner, TokenOwner.moderator) # Create a new workflow with the previous workflow's save token, and # restore its state. This models an approved un-sunscription request # and should result in the user getting subscribed. approved_workflow = UnSubscriptionWorkflow(self._mlist) approved_workflow.token = workflow.token approved_workflow.restore() list(approved_workflow) # Now the user is unsubscribed from the mailing list. member = self._mlist.regular_members.get_member(self._anne) self.assertIsNone(member) self.assertEqual(approved_workflow.member, member) # No further token is needed. self.assertIsNone(approved_workflow.token) self.assertEqual(approved_workflow.token_owner, TokenOwner.no_one) def test_get_moderator_approval_log_on_hold(self): # When the unsubscription is held for moderator approval, a message is # logged. mark = LogFileMark('mailman.subscribe') self._mlist.unsubscription_policy = SubscriptionPolicy.moderate workflow = UnSubscriptionWorkflow( self._mlist, self.anne, pre_confirmed=True) # Run the entire workflow. list(workflow) self.assertIn( 'test@example.com: held unsubscription request from anne@example.com', mark.readline() ) # The state machine stopped at the moderator approval step so there # will be one token still in the database. self._expected_pendings_count = 1 def test_get_moderator_approval_notifies_moderators(self): # When the unsubscription is held for moderator approval, and the list # is so configured, a notification is sent to the list moderators. self._mlist.admin_immed_notify = True self._mlist.unsubscription_policy = SubscriptionPolicy.moderate workflow = UnSubscriptionWorkflow( self._mlist, self.anne, pre_confirmed=True) # Consume the entire state machine. list(workflow) items = get_queue_messages('virgin', expected_count=1) message = items[0].msg self.assertEqual(message['From'], 'test-owner@example.com') self.assertEqual(message['To'], 'test-owner@example.com') self.assertEqual( message['Subject'], 'New unsubscription request to Test from anne@example.com') self.assertEqual(message.get_payload(), """\ Your authorization is required for a mailing list unsubscription request approval: For: anne@example.com List: test@example.com """) # The state machine stopped at the moderator approval so there will be # one token still in the database. self._expected_pendings_count = 1 def test_get_moderator_approval_no_notifications(self): # When the unsubscription request is held for moderator approval, and # the list is so configured, a notification is sent to the list # moderators. self._mlist.admin_immed_notify = False self._mlist.unsubscription_policy = SubscriptionPolicy.moderate workflow = UnSubscriptionWorkflow( self._mlist, self.anne, pre_confirmed=True) # Consume the entire state machine. list(workflow) get_queue_messages('virgin', expected_count=0) # The state machine stopped at the moderator approval so there will be # one token still in the database. self._expected_pendings_count = 1 def test_send_confirmation(self): # A confirmation message gets sent when the unsubscription must be # confirmed. self._mlist.unsubscription_policy = SubscriptionPolicy.confirm # Run the workflow to model the confirmation step. workflow = UnSubscriptionWorkflow(self._mlist, self.anne) list(workflow) items = get_queue_messages('virgin', expected_count=1) message = items[0].msg token = workflow.token self.assertEqual( message['Subject'], 'confirm {}'.format(workflow.token)) self.assertEqual( message['From'], 'test-confirm+{}@example.com'.format(token)) # The state machine stopped at the member confirmation step so there # will be one token still in the database. self._expected_pendings_count = 1 def test_do_confirmation_unsubscribes_user(self): # Unsubscriptions to the mailing list must be confirmed. Once that's # done, the user's address is unsubscribed. self._mlist.unsubscription_policy = SubscriptionPolicy.confirm workflow = UnSubscriptionWorkflow(self._mlist, self.anne) list(workflow) # Anne is a member. member = self._mlist.regular_members.get_member(self._anne) self.assertIsNotNone(member) self.assertEqual(member, workflow.member) # The token is owned by the subscriber. self.assertIsNotNone(workflow.token) self.assertEqual(workflow.token_owner, TokenOwner.subscriber) # Confirm. confirm_workflow = UnSubscriptionWorkflow(self._mlist) confirm_workflow.token = workflow.token confirm_workflow.restore() list(confirm_workflow) # Anne is now unsubscribed. member = self._mlist.regular_members.get_member(self._anne) self.assertIsNone(member) # No further token is needed. self.assertIsNone(confirm_workflow.token) self.assertEqual(confirm_workflow.token_owner, TokenOwner.no_one) def test_do_confirmation_unsubscribes_address(self): # Unsubscriptions to the mailing list must be confirmed. Once that's # done, the address is unsubscribed. address = self.anne.register('anne.person@example.com') self._mlist.subscribe(address) self._mlist.unsubscription_policy = SubscriptionPolicy.confirm workflow = UnSubscriptionWorkflow(self._mlist, address) list(workflow) # Bart is a member. member = self._mlist.regular_members.get_member( 'anne.person@example.com') self.assertIsNotNone(member) self.assertEqual(member, workflow.member) # The token is owned by the subscriber. self.assertIsNotNone(workflow.token) self.assertEqual(workflow.token_owner, TokenOwner.subscriber) # Confirm. confirm_workflow = UnSubscriptionWorkflow(self._mlist) confirm_workflow.token = workflow.token confirm_workflow.restore() list(confirm_workflow) # Bart is now unsubscribed. member = self._mlist.regular_members.get_member( 'anne.person@example.com') self.assertIsNone(member) # No further token is needed. self.assertIsNone(confirm_workflow.token) self.assertEqual(confirm_workflow.token_owner, TokenOwner.no_one) def test_do_confirmation_nonmember(self): # Attempt to confirm the unsubscription of a member who has already # been unsubscribed. self._mlist.unsubscription_policy = SubscriptionPolicy.confirm workflow = UnSubscriptionWorkflow(self._mlist, self.anne) list(workflow) # Anne is a member. member = self._mlist.regular_members.get_member(self._anne) self.assertIsNotNone(member) self.assertEqual(member, workflow.member) # The token is owned by the subscriber. self.assertIsNotNone(workflow.token) self.assertEqual(workflow.token_owner, TokenOwner.subscriber) # Unsubscribe Anne out of band. member.unsubscribe() # Confirm. confirm_workflow = UnSubscriptionWorkflow(self._mlist) confirm_workflow.token = workflow.token confirm_workflow.restore() list(confirm_workflow) # No further token is needed. self.assertIsNone(confirm_workflow.token) self.assertEqual(confirm_workflow.token_owner, TokenOwner.no_one) def test_do_confirmation_nonmember_final_step(self): # Attempt to confirm the unsubscription of a member who has already # been unsubscribed. self._mlist.unsubscription_policy = SubscriptionPolicy.confirm workflow = UnSubscriptionWorkflow(self._mlist, self.anne) list(workflow) # Anne is a member. member = self._mlist.regular_members.get_member(self._anne) self.assertIsNotNone(member) self.assertEqual(member, workflow.member) # The token is owned by the subscriber. self.assertIsNotNone(workflow.token) self.assertEqual(workflow.token_owner, TokenOwner.subscriber) # Confirm. confirm_workflow = UnSubscriptionWorkflow(self._mlist) confirm_workflow.token = workflow.token confirm_workflow.restore() confirm_workflow.run_until('do_unsubscription') self.assertEqual(member, confirm_workflow.member) # Unsubscribe Anne out of band. member.unsubscribe() list(confirm_workflow) self.assertIsNone(confirm_workflow.member) # No further token is needed. self.assertIsNone(confirm_workflow.token) self.assertEqual(confirm_workflow.token_owner, TokenOwner.no_one) def test_prevent_confirmation_replay_attacks(self): # Ensure that if the workflow requires two confirmations, e.g. first # the user confirming their subscription, and then the moderator # approving it, that different tokens are used in these two cases. self._mlist.unsubscription_policy = ( SubscriptionPolicy.confirm_then_moderate) workflow = UnSubscriptionWorkflow(self._mlist, self.anne) # Run the state machine up to the first confirmation, and cache the # confirmation token. list(workflow) token = workflow.token # Anne is still a member of the mailing list. member = self._mlist.regular_members.get_member(self._anne) self.assertIsNotNone(member) self.assertIsNotNone(workflow.member) # The token is owned by the subscriber. self.assertIsNotNone(workflow.token) self.assertEqual(workflow.token_owner, TokenOwner.subscriber) # The old token will not work for moderator approval. moderator_workflow = UnSubscriptionWorkflow(self._mlist) moderator_workflow.token = token moderator_workflow.restore() list(moderator_workflow) # The token is owned by the moderator. self.assertIsNotNone(moderator_workflow.token) self.assertEqual(moderator_workflow.token_owner, TokenOwner.moderator) # While we wait for the moderator to approve the subscription, note # that there's a new token for the next steps. self.assertNotEqual(token, moderator_workflow.token) # The old token won't work. final_workflow = UnSubscriptionWorkflow(self._mlist) final_workflow.token = token self.assertRaises(LookupError, final_workflow.restore) # Running this workflow will fail. self.assertRaises(AssertionError, list, final_workflow) # Anne is still not unsubscribed. member = self._mlist.regular_members.get_member(self._anne) self.assertIsNotNone(member) self.assertIsNone(final_workflow.member) # However, if we use the new token, her unsubscription request will be # approved by the moderator. final_workflow.token = moderator_workflow.token final_workflow.restore() list(final_workflow) # And now Anne is unsubscribed. member = self._mlist.regular_members.get_member(self._anne) self.assertIsNone(member) # No further token is needed. self.assertIsNone(final_workflow.token) self.assertEqual(final_workflow.token_owner, TokenOwner.no_one) def test_confirmation_needed_and_pre_confirmed(self): # The subscription policy is 'confirm' but the subscription is # pre-confirmed so the moderation checks can be skipped. self._mlist.unsubscription_policy = SubscriptionPolicy.confirm workflow = UnSubscriptionWorkflow( self._mlist, self.anne, pre_confirmed=True, pre_approved=True) list(workflow) # Anne was unsubscribed. self.assertIsNone(workflow.token) self.assertEqual(workflow.token_owner, TokenOwner.no_one) self.assertIsNone(workflow.member) def test_confirmation_needed_moderator_address(self): address = self.anne.register('anne.person@example.com') self._mlist.subscribe(address) self._mlist.unsubscription_policy = SubscriptionPolicy.moderate workflow = UnSubscriptionWorkflow(self._mlist, address) # Get moderator approval. list(workflow) approved_workflow = UnSubscriptionWorkflow(self._mlist) approved_workflow.token = workflow.token approved_workflow.restore() list(approved_workflow) self.assertEqual(approved_workflow.subscriber, address) # Anne was unsubscribed. self.assertIsNone(approved_workflow.token) self.assertEqual(approved_workflow.token_owner, TokenOwner.no_one) self.assertIsNone(approved_workflow.member) member = self._mlist.regular_members.get_member( 'anne.person@example.com') self.assertIsNone(member) mailman-3.2.2/src/mailman/app/tests/test_workflow.py0000644000175000017500000001411013442110351023653 0ustar maxkingmaxking00000000000000# Copyright (C) 2015-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """App-level workflow tests.""" import json import unittest from mailman.app.workflow import Workflow from mailman.interfaces.workflow import IWorkflowStateManager from mailman.testing.layers import ConfigLayer from zope.component import getUtility class MyWorkflow(Workflow): INITIAL_STATE = 'first' SAVE_ATTRIBUTES = ('ant', 'bee', 'cat') def __init__(self): super().__init__() self.token = 'test-workflow' self.ant = 1 self.bee = 2 self.cat = 3 self.dog = 4 def _step_first(self): self.push('second') return 'one' def _step_second(self): self.push('third') return 'two' def _step_third(self): return 'three' class DependentWorkflow(MyWorkflow): SAVE_ATTRIBUTES = ('ant', 'bee', 'cat', 'elf') def __init__(self): super().__init__() self._elf = 5 @property def elf(self): return self._elf @elf.setter def elf(self, value): # This attribute depends on other attributes. assert self.ant is not None assert self.bee is not None assert self.cat is not None self._elf = value class TestWorkflow(unittest.TestCase): layer = ConfigLayer def setUp(self): self._workflow = iter(MyWorkflow()) def test_basic_workflow(self): # The work flows from one state to the next. results = list(self._workflow) self.assertEqual(results, ['one', 'two', 'three']) def test_partial_workflow(self): # You don't have to flow through every step. results = next(self._workflow) self.assertEqual(results, 'one') def test_exhaust_workflow(self): # Manually flow through a few steps, then consume the whole thing. results = [next(self._workflow)] results.extend(self._workflow) self.assertEqual(results, ['one', 'two', 'three']) def test_save_and_restore_workflow(self): # Without running any steps, save and restore the workflow. Then # consume the restored workflow. self._workflow.save() new_workflow = MyWorkflow() new_workflow.restore() results = list(new_workflow) self.assertEqual(results, ['one', 'two', 'three']) def test_save_and_restore_partial_workflow(self): # After running a few steps, save and restore the workflow. Then # consume the restored workflow. next(self._workflow) self._workflow.save() new_workflow = MyWorkflow() new_workflow.restore() results = list(new_workflow) self.assertEqual(results, ['two', 'three']) def test_save_and_restore_exhausted_workflow(self): # After consuming the entire workflow, save and restore it. list(self._workflow) self._workflow.save() new_workflow = MyWorkflow() new_workflow.restore() results = list(new_workflow) self.assertEqual(len(results), 0) def test_save_and_restore_attributes(self): # Saved attributes are restored. self._workflow.ant = 9 self._workflow.bee = 8 self._workflow.cat = 7 # Don't save .dog. self._workflow.save() new_workflow = MyWorkflow() new_workflow.restore() self.assertEqual(new_workflow.ant, 9) self.assertEqual(new_workflow.bee, 8) self.assertEqual(new_workflow.cat, 7) self.assertEqual(new_workflow.dog, 4) def test_save_and_restore_dependant_attributes(self): # Attributes must be restored in the order they are declared in # SAVE_ATTRIBUTES. workflow = iter(DependentWorkflow()) workflow.elf = 6 workflow.save() new_workflow = DependentWorkflow() # The elf attribute must be restored last, set triggering values for # attributes it depends on. new_workflow.ant = new_workflow.bee = new_workflow.cat = None new_workflow.restore() self.assertEqual(new_workflow.elf, 6) def test_save_and_restore_obsolete_attributes(self): # Obsolete saved attributes are ignored. state_manager = getUtility(IWorkflowStateManager) # Save the state of an old version of the workflow that would not have # the cat attribute. state_manager.save( self._workflow.token, 'first', json.dumps({'ant': 1, 'bee': 2})) # Restore in the current version that needs the cat attribute. new_workflow = MyWorkflow() try: new_workflow.restore() except KeyError: self.fail('Restore does not handle obsolete attributes') # Restoring must not raise an exception, the default value is kept. self.assertEqual(new_workflow.cat, 3) def test_run_thru(self): # Run all steps through the given one. results = self._workflow.run_thru('second') self.assertEqual(results, ['one', 'two']) def test_run_thru_completes(self): results = self._workflow.run_thru('all of them') self.assertEqual(results, ['one', 'two', 'three']) def test_run_until(self): # Run until (but not including) the given step. results = self._workflow.run_until('second') self.assertEqual(results, ['one']) def test_run_until_completes(self): results = self._workflow.run_until('all of them') self.assertEqual(results, ['one', 'two', 'three']) mailman-3.2.2/src/mailman/app/tests/test_workflowmanager.py0000644000175000017500000003232013442110351025211 0ustar maxkingmaxking00000000000000# Copyright (C) 2012-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Test email address registration.""" import unittest from mailman.app.lifecycle import create_list from mailman.interfaces.mailinglist import SubscriptionPolicy from mailman.interfaces.member import MemberRole from mailman.interfaces.pending import IPendings from mailman.interfaces.subscriptions import ISubscriptionManager, TokenOwner from mailman.interfaces.usermanager import IUserManager from mailman.testing.helpers import get_queue_messages from mailman.testing.layers import ConfigLayer from mailman.utilities.datetime import now from zope.component import getUtility class TestRegistrar(unittest.TestCase): """Test registration.""" layer = ConfigLayer def setUp(self): self._mlist = create_list('ant@example.com') self._registrar = ISubscriptionManager(self._mlist) self._pendings = getUtility(IPendings) self._anne = getUtility(IUserManager).create_address( 'anne@example.com') def test_initial_conditions(self): # Registering a subscription request provides a unique token associated # with a pendable, and the owner of the token. self.assertEqual(self._pendings.count, 0) token, token_owner, member = self._registrar.register(self._anne) self.assertIsNotNone(token) self.assertEqual(token_owner, TokenOwner.subscriber) self.assertIsNone(member) self.assertEqual(self._pendings.count, 1) record = self._pendings.confirm(token, expunge=False) self.assertEqual(record['list_id'], self._mlist.list_id) self.assertEqual(record['email'], 'anne@example.com') def test_subscribe(self): # Registering a subscription request where no confirmation or # moderation steps are needed, leaves us with no token or owner, since # there's nothing more to do. self._mlist.subscription_policy = SubscriptionPolicy.open self._anne.verified_on = now() token, token_owner, rmember = self._registrar.register(self._anne) self.assertIsNone(token) self.assertEqual(token_owner, TokenOwner.no_one) member = self._mlist.regular_members.get_member('anne@example.com') self.assertEqual(rmember, member) self.assertEqual(member.address, self._anne) # There's nothing to confirm. record = self._pendings.confirm(token, expunge=False) self.assertIsNone(record) def test_no_such_token(self): # Given a token which is not in the database, a LookupError is raised. self._registrar.register(self._anne) self.assertRaises(LookupError, self._registrar.confirm, 'not-a-token') def test_confirm_because_verify(self): # We have a subscription request which requires the user to confirm # (because she does not have a verified address), but not the moderator # to approve. Running the workflow gives us a token. Confirming the # token subscribes the user. self._mlist.subscription_policy = SubscriptionPolicy.open token, token_owner, rmember = self._registrar.register(self._anne) self.assertIsNotNone(token) self.assertEqual(token_owner, TokenOwner.subscriber) self.assertIsNone(rmember) member = self._mlist.regular_members.get_member('anne@example.com') self.assertIsNone(member) # Now confirm the subscription. token, token_owner, rmember = self._registrar.confirm(token) self.assertIsNone(token) self.assertEqual(token_owner, TokenOwner.no_one) member = self._mlist.regular_members.get_member('anne@example.com') self.assertEqual(rmember, member) self.assertEqual(member.address, self._anne) def test_confirm_because_confirm(self): # We have a subscription request which requires the user to confirm # (because of list policy), but not the moderator to approve. Running # the workflow gives us a token. Confirming the token subscribes the # user. self._mlist.subscription_policy = SubscriptionPolicy.confirm self._anne.verified_on = now() token, token_owner, rmember = self._registrar.register(self._anne) self.assertIsNotNone(token) self.assertEqual(token_owner, TokenOwner.subscriber) self.assertIsNone(rmember) member = self._mlist.regular_members.get_member('anne@example.com') self.assertIsNone(member) # Now confirm the subscription. token, token_owner, rmember = self._registrar.confirm(token) self.assertIsNone(token) self.assertEqual(token_owner, TokenOwner.no_one) member = self._mlist.regular_members.get_member('anne@example.com') self.assertEqual(rmember, member) self.assertEqual(member.address, self._anne) def test_confirm_because_moderation(self): # We have a subscription request which requires the moderator to # approve. Running the workflow gives us a token. Confirming the # token subscribes the user. self._mlist.subscription_policy = SubscriptionPolicy.moderate self._anne.verified_on = now() token, token_owner, rmember = self._registrar.register(self._anne) self.assertIsNotNone(token) self.assertEqual(token_owner, TokenOwner.moderator) self.assertIsNone(rmember) member = self._mlist.regular_members.get_member('anne@example.com') self.assertIsNone(member) # Now confirm the subscription. token, token_owner, rmember = self._registrar.confirm(token) self.assertIsNone(token) self.assertEqual(token_owner, TokenOwner.no_one) member = self._mlist.regular_members.get_member('anne@example.com') self.assertEqual(rmember, member) self.assertEqual(member.address, self._anne) def test_confirm_because_confirm_then_moderation(self): # We have a subscription request which requires the user to confirm # (because she does not have a verified address) and the moderator to # approve. Running the workflow gives us a token. Confirming the # token runs the workflow a little farther, but still gives us a # token. Confirming again subscribes the user. self._mlist.subscription_policy = ( SubscriptionPolicy.confirm_then_moderate) self._anne.verified_on = now() # Runs until subscription confirmation. token, token_owner, rmember = self._registrar.register(self._anne) self.assertIsNotNone(token) self.assertEqual(token_owner, TokenOwner.subscriber) self.assertIsNone(rmember) member = self._mlist.regular_members.get_member('anne@example.com') self.assertIsNone(member) # Now confirm the subscription, and wait for the moderator to approve # the subscription. She is still not subscribed. new_token, token_owner, rmember = self._registrar.confirm(token) # The new token, used for the moderator to approve the message, is not # the same as the old token. self.assertNotEqual(new_token, token) self.assertIsNotNone(new_token) self.assertEqual(token_owner, TokenOwner.moderator) self.assertIsNone(rmember) member = self._mlist.regular_members.get_member('anne@example.com') self.assertIsNone(member) # Confirm once more, this time as the moderator approving the # subscription. Now she's a member. token, token_owner, rmember = self._registrar.confirm(new_token) self.assertIsNone(token) self.assertEqual(token_owner, TokenOwner.no_one) member = self._mlist.regular_members.get_member('anne@example.com') self.assertEqual(rmember, member) self.assertEqual(member.address, self._anne) def test_confirm_then_moderate_with_different_tokens(self): # Ensure that the confirmation token the user sees when they have to # confirm their subscription is different than the token the moderator # sees when they approve the subscription. This prevents the user # from using a replay attack to subvert moderator approval. self._mlist.subscription_policy = ( SubscriptionPolicy.confirm_then_moderate) self._anne.verified_on = now() # Runs until subscription confirmation. token, token_owner, rmember = self._registrar.register(self._anne) self.assertIsNotNone(token) self.assertEqual(token_owner, TokenOwner.subscriber) self.assertIsNone(rmember) member = self._mlist.regular_members.get_member('anne@example.com') self.assertIsNone(member) # Now confirm the subscription, and wait for the moderator to approve # the subscription. She is still not subscribed. new_token, token_owner, rmember = self._registrar.confirm(token) # The status is not true because the user has not yet been subscribed # to the mailing list. self.assertIsNotNone(new_token) self.assertEqual(token_owner, TokenOwner.moderator) self.assertIsNone(rmember) member = self._mlist.regular_members.get_member('anne@example.com') self.assertIsNone(member) # The new token is different than the old token. self.assertNotEqual(token, new_token) # Trying to confirm with the old token does not work. self.assertRaises(LookupError, self._registrar.confirm, token) # Confirm once more, this time with the new token, as the moderator # approving the subscription. Now she's a member. done_token, token_owner, rmember = self._registrar.confirm(new_token) # The token is None, signifying that the member has been subscribed. self.assertIsNone(done_token) self.assertEqual(token_owner, TokenOwner.no_one) member = self._mlist.regular_members.get_member('anne@example.com') self.assertEqual(rmember, member) self.assertEqual(member.address, self._anne) def test_discard_waiting_for_confirmation(self): # While waiting for a user to confirm their subscription, we discard # the workflow. self._mlist.subscription_policy = SubscriptionPolicy.confirm self._anne.verified_on = now() # Runs until subscription confirmation. token, token_owner, rmember = self._registrar.register(self._anne) self.assertIsNotNone(token) self.assertEqual(token_owner, TokenOwner.subscriber) self.assertIsNone(rmember) member = self._mlist.regular_members.get_member('anne@example.com') self.assertIsNone(member) # Now discard the subscription request. self._registrar.discard(token) # Trying to confirm the token now results in an exception. self.assertRaises(LookupError, self._registrar.confirm, token) def test_admin_notify_mchanges(self): # When a user gets subscribed via the subscription policy workflow, # the list administrators get an email notification. self._mlist.subscription_policy = SubscriptionPolicy.open self._mlist.admin_notify_mchanges = True self._mlist.send_welcome_message = False token, token_owner, member = self._registrar.register( self._anne, pre_verified=True) # Anne is now a member. self.assertEqual(member.address.email, 'anne@example.com') # And there's a notification email waiting for Bart. items = get_queue_messages('virgin', expected_count=1) message = items[0].msg self.assertEqual(message['To'], 'ant-owner@example.com') self.assertEqual(message['Subject'], 'Ant subscription notification') self.assertEqual(message.get_payload(), """\ anne@example.com has been successfully subscribed to Ant. """) def test_no_admin_notify_mchanges(self): # Even when a user gets subscribed via the subscription policy # workflow, the list administrators won't get an email notification if # they don't want one. self._mlist.subscription_policy = SubscriptionPolicy.open self._mlist.admin_notify_mchanges = False self._mlist.send_welcome_message = False # Bart is an administrator of the mailing list. bart = getUtility(IUserManager).create_address( 'bart@example.com', 'Bart Person') self._mlist.subscribe(bart, MemberRole.owner) token, token_owner, member = self._registrar.register( self._anne, pre_verified=True) # Anne is now a member. self.assertEqual(member.address.email, 'anne@example.com') # There's no notification email waiting for Bart. get_queue_messages('virgin', expected_count=0) mailman-3.2.2/src/mailman/app/workflow.py0000644000175000017500000001165013442110351021460 0ustar maxkingmaxking00000000000000# Copyright (C) 2015-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Generic workflow.""" import sys import json import logging from collections import deque from mailman.interfaces.workflow import IWorkflowStateManager from public import public from zope.component import getUtility COMMASPACE = ', ' log = logging.getLogger('mailman.error') @public class Workflow: """Generic workflow.""" SAVE_ATTRIBUTES = () INITIAL_STATE = None def __init__(self): self.token = None self._next = deque() self.push(self.INITIAL_STATE) self.debug = False self._count = 0 @property def name(self): return self.__class__.__name__ def __iter__(self): return self def push(self, step): self._next.append(step) def _pop(self): name = self._next.popleft() step = getattr(self, '_step_{}'.format(name)) self._count += 1 if self.debug: # pragma: nocover print('[{:02d}] -> {}'.format(self._count, name), file=sys.stderr) return name, step def __next__(self): try: name, step = self._pop() return step() except IndexError: raise StopIteration except: # noqa: E722 log.exception('deque: {}'.format(COMMASPACE.join(self._next))) raise def run_thru(self, stop_after): """Run the state machine through and including the given step. :param stop_after: Name of method, sans prefix to run the state machine through. In other words, the state machine runs until the named method completes. """ results = [] while True: try: name, step = self._pop() except (StopIteration, IndexError): # We're done. break results.append(step()) if name == stop_after: break return results def run_until(self, stop_before): """Trun the state machine until (not including) the given step. :param stop_before: Name of method, sans prefix that the state machine is run until the method is reached. Unlike `run_thru()` the named method is not run. """ results = [] while True: try: name, step = self._pop() except (StopIteration, IndexError): # We're done. break if name == stop_before: # Stop executing, but not before we push the last state back # onto the deque. Otherwise, resuming the state machine would # skip this step. self._next.appendleft(name) break results.append(step()) return results def save(self): assert self.token, 'Workflow token must be set' state_manager = getUtility(IWorkflowStateManager) data = {attr: getattr(self, attr) for attr in self.SAVE_ATTRIBUTES} # Note: only the next step is saved, not the whole stack. This is not # an issue in practice, since there's never more than a single step in # the queue anyway. If we want to support more than a single step in # the queue *and* want to support state saving/restoring, change this # method and the restore() method. if len(self._next) == 0: step = None elif len(self._next) == 1: step = self._next[0] else: raise AssertionError( "Can't save a workflow state with more than one step " "in the queue") state_manager.save(self.token, step, json.dumps(data)) def restore(self): state_manager = getUtility(IWorkflowStateManager) state = state_manager.restore(self.token) if state is None: # The token doesn't exist in the database. raise LookupError(self.token) self._next.clear() if state.step: self._next.append(state.step) data = json.loads(state.data) for attr in self.SAVE_ATTRIBUTES: try: setattr(self, attr, data[attr]) except KeyError: pass mailman-3.2.2/src/mailman/archiving/0000755000175000017500000000000013445614541020440 5ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/archiving/__init__.py0000644000175000017500000000000013244427337022541 0ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/archiving/docs/0000755000175000017500000000000013445614541021370 5ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/archiving/docs/__init__.py0000644000175000017500000000000013244427337023471 0ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/archiving/docs/common.rst0000644000175000017500000001332613442110351023402 0ustar maxkingmaxking00000000000000========= Archivers ========= Mailman supports pluggable archivers, and it comes with several default archivers. >>> mlist = create_list('test@example.com') >>> msg = message_from_string("""\ ... From: aperson@example.org ... To: test@example.com ... Subject: An archived message ... Message-ID: <12345> ... X-Message-ID-Hash: RSZCG7IGPHFIRW3EMTVMMDNJMNCVCOLE ... ... Here is an archived message. ... """) Archivers support an interface which provides the RFC 2369 ``List-Archive:`` header, and one that provides a *permalink* to the specific message object in the archive. This latter is appropriate for the message footer or for the RFC 5064 ``Archived-At:`` header. If the archiver is not network-accessible, it will return ``None`` and the headers will not be added. Mailman defines a draft spec for how list servers and archivers can interoperate. >>> archivers = {} >>> from operator import attrgetter >>> for archiver in sorted(config.archivers, key=attrgetter('name')): ... print(archiver.name) ... print(' ', archiver.list_url(mlist)) ... print(' ', archiver.permalink(mlist, msg)) ... archivers[archiver.name] = archiver mail-archive http://go.mail-archive.dev/test%40example.com http://go.mail-archive.dev/RSZCG7IGPHFIRW3EMTVMMDNJMNCVCOLE mhonarc http://example.com/.../test@example.com http://example.com/.../RSZCG7IGPHFIRW3EMTVMMDNJMNCVCOLE prototype None None Sending the message to the archiver =================================== The `prototype` archiver archives messages to a maildir. >>> import os >>> archivers['prototype'].archive_message(mlist, msg) >>> archive_path = os.path.join( ... config.ARCHIVE_DIR, 'prototype', mlist.fqdn_listname, 'new') >>> len(os.listdir(archive_path)) 1 The Mail-Archive.com ==================== `The Mail Archive`_ is a public archiver that can be used to archive message for free. Mailman comes with a plugin for this archiver; by enabling it messages to public lists will get sent there automatically. >>> archiver = archivers['mail-archive'] >>> print(archiver.list_url(mlist)) http://go.mail-archive.dev/test%40example.com >>> print(archiver.permalink(mlist, msg)) http://go.mail-archive.dev/RSZCG7IGPHFIRW3EMTVMMDNJMNCVCOLE To archive the message, the archiver actually mails the message to a special address at The Mail Archive. The message gets no header or footer decoration. :: >>> from mailman.interfaces.archiver import ArchivePolicy >>> mlist.archive_policy = ArchivePolicy.public >>> archiver.archive_message(mlist, msg) >>> from mailman.runners.outgoing import OutgoingRunner >>> from mailman.testing.helpers import make_testable_runner >>> outgoing = make_testable_runner(OutgoingRunner, 'out') >>> outgoing.run() >>> from operator import itemgetter >>> messages = list(smtpd.messages) >>> len(messages) 1 >>> print(messages[0].as_string()) From: aperson@example.org To: test@example.com Subject: An archived message Message-ID: <12345> X-Message-ID-Hash: RSZCG7IGPHFIRW3EMTVMMDNJMNCVCOLE X-Peer: ... X-MailFrom: test-bounces@example.com X-RcptTo: archive@mail-archive.dev Here is an archived message. >>> smtpd.clear() However, if the mailing list is not public, the message will never be archived at this service. >>> mlist.archive_policy = ArchivePolicy.private >>> print(archiver.list_url(mlist)) None >>> print(archiver.permalink(mlist, msg)) None >>> archiver.archive_message(mlist, msg) >>> list(smtpd.messages) [] Additionally, this archiver can handle malformed ``Message-IDs``. :: >>> from mailman.utilities.email import add_message_hash >>> mlist.archive_policy = ArchivePolicy.public >>> del msg['message-id'] >>> del msg['x-message-id-hash'] >>> msg['Message-ID'] = '12345>' >>> add_message_hash(msg) 'YJIGBYRWZFG5LZEBQ7NR25B5HBR2BVD6' >>> print(archiver.permalink(mlist, msg)) http://go.mail-archive.dev/YJIGBYRWZFG5LZEBQ7NR25B5HBR2BVD6 >>> del msg['message-id'] >>> del msg['x-message-id-hash'] >>> msg['Message-ID'] = '<12345' >>> add_message_hash(msg) 'XUFFJNJ2P2WC4NDPQRZFDJMV24POP64B' >>> print(archiver.permalink(mlist, msg)) http://go.mail-archive.dev/XUFFJNJ2P2WC4NDPQRZFDJMV24POP64B >>> del msg['message-id'] >>> del msg['x-message-id-hash'] >>> msg['Message-ID'] = '12345' >>> add_message_hash(msg) 'RSZCG7IGPHFIRW3EMTVMMDNJMNCVCOLE' >>> print(archiver.permalink(mlist, msg)) http://go.mail-archive.dev/RSZCG7IGPHFIRW3EMTVMMDNJMNCVCOLE >>> del msg['message-id'] >>> del msg['x-message-id-hash'] >>> add_message_hash(msg) >>> msg['Message-ID'] = ' 12345 ' >>> add_message_hash(msg) 'RSZCG7IGPHFIRW3EMTVMMDNJMNCVCOLE' >>> print(archiver.permalink(mlist, msg)) http://go.mail-archive.dev/RSZCG7IGPHFIRW3EMTVMMDNJMNCVCOLE MHonArc ======= A MHonArc_ archiver is also available. >>> archiver = archivers['mhonarc'] >>> print(archiver.name) mhonarc Messages sent to a local MHonArc instance are added to its archive via a subprocess call. >>> from mailman.testing.helpers import LogFileMark >>> mark = LogFileMark('mailman.archiver') >>> archiver.archive_message(mlist, msg) >>> print('LOG:', mark.readline()) LOG: ... /usr/bin/mhonarc -add -dbfile .../test@example.com.mbox/mhonarc.db -outdir .../mhonarc/test@example.com -stderr .../logs/mhonarc -stdout .../logs/mhonarc -spammode -umask 022 .. _`The Mail Archive`: https://www.mail-archive.com .. _MHonArc: https://www.mhonarc.org mailman-3.2.2/src/mailman/archiving/mailarchive.py0000644000175000017500000000547313442110351023272 0ustar maxkingmaxking00000000000000# Copyright (C) 2008-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """The Mail-Archive.com archiver.""" from mailman.config import config from mailman.config.config import external_configuration from mailman.interfaces.archiver import ArchivePolicy, IArchiver from public import public from urllib.parse import quote, urljoin from zope.interface import implementer @public @implementer(IArchiver) class MailArchive: """Public archiver at the Mail-Archive.com. Messages get archived at http://go.mail-archive.com. """ name = 'mail-archive' is_enabled = False def __init__(self): # Read our specific configuration file archiver_config = external_configuration( config.archiver.mail_archive.configuration) self.base_url = archiver_config.get('general', 'base_url') self.recipient = archiver_config.get('general', 'recipient') def list_url(self, mlist): """See `IArchiver`.""" if mlist.archive_policy is ArchivePolicy.public: return urljoin(self.base_url, quote(mlist.posting_address)) return None def permalink(self, mlist, msg): """See `IArchiver`.""" if mlist.archive_policy is not ArchivePolicy.public: return None # It is the LMTP server's responsibility to ensure that the message has # a Message-ID-Hash header. For backward compatibility, fallback to # searching for X-Message-ID-Hash. If the message has neither, then # there's no permalink. message_id_hash = msg.get('message-id-hash') if message_id_hash is None: message_id_hash = msg.get('x-message-id-hash') if message_id_hash is None: return None if isinstance(message_id_hash, bytes): message_id_hash = message_id_hash.decode('ascii') return urljoin(self.base_url, message_id_hash) def archive_message(self, mlist, msg): """See `IArchiver`.""" if mlist.archive_policy is ArchivePolicy.public: config.switchboards['out'].enqueue( msg, listid=mlist.list_id, recipients=[self.recipient]) return None mailman-3.2.2/src/mailman/archiving/mhonarc.py0000644000175000017500000000656113442110351022434 0ustar maxkingmaxking00000000000000# Copyright (C) 2008-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """MHonArc archiver.""" import logging from mailman.config import config from mailman.config.config import external_configuration from mailman.interfaces.archiver import IArchiver from mailman.utilities.string import expand from public import public from subprocess import PIPE, Popen from urllib.parse import urljoin from zope.interface import implementer log = logging.getLogger('mailman.archiver') @public @implementer(IArchiver) class MHonArc: """Local MHonArc archiver.""" name = 'mhonarc' is_enabled = False def __init__(self): # Read our specific configuration file archiver_config = external_configuration( config.archiver.mhonarc.configuration) self.base_url = archiver_config.get('general', 'base_url') self.command = archiver_config.get('general', 'command') def list_url(self, mlist): """See `IArchiver`.""" # XXX What about private MHonArc archives? return expand(self.base_url, mlist, dict( # For backward compatibility. hostname=mlist.domain.mail_host, fqdn_listname=mlist.fqdn_listname, )) def permalink(self, mlist, msg): """See `IArchiver`.""" # XXX What about private MHonArc archives? # # It is the LMTP server's responsibility to ensure that the message has # a Message-ID-Hash header. For backward compatibility, fall back to # X-Message-ID-Hash. If the message has neither, then there's no # permalink. message_id_hash = msg.get('message-id-hash') if message_id_hash is None: message_id_hash = msg.get('x-message-id-hash') if message_id_hash is None: return None if isinstance(message_id_hash, bytes): message_id_hash = message_id_hash.decode('ascii') return urljoin(self.list_url(mlist), message_id_hash) def archive_message(self, mlist, msg): """See `IArchiver`.""" substitutions = config.__dict__.copy() substitutions['listname'] = mlist.fqdn_listname command = expand(self.command, mlist, substitutions) proc = Popen( command, stdin=PIPE, stdout=PIPE, stderr=PIPE, universal_newlines=True, shell=True) stdout, stderr = proc.communicate(msg.as_string()) if proc.returncode != 0: log.error('%s: mhonarc subprocess had non-zero exit code: %s' % (msg['message-id'], proc.returncode)) log.info(stdout) log.error(stderr) # Can we get more information, such as the url to the message just # archived, out of MHonArc? return None mailman-3.2.2/src/mailman/archiving/prototype.py0000644000175000017500000000746213442110351023053 0ustar maxkingmaxking00000000000000# Copyright (C) 2008-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Prototypical permalinking archiver.""" import os import logging from contextlib import suppress from datetime import timedelta from flufl.lock import Lock, TimeOutError from mailbox import Maildir from mailman.config import config from mailman.interfaces.archiver import IArchiver from public import public from zope.interface import implementer log = logging.getLogger('mailman.error') @public @implementer(IArchiver) class Prototype: """A prototype of a third party archiver. Mailman proposes a draft specification for interoperability between list servers and archivers: . """ name = 'prototype' is_enabled = False @staticmethod def list_url(mlist): """See `IArchiver`.""" # This archiver is not web-accessible, therefore no URL is returned. return None @staticmethod def permalink(mlist, msg): """See `IArchiver`.""" # This archiver is not web-accessible, therefore no URL is returned. return None @staticmethod def archive_message(mlist, message): """See `IArchiver`. This archiver saves messages into a maildir. """ archive_dir = os.path.join(config.ARCHIVE_DIR, 'prototype') with suppress(FileExistsError): os.makedirs(archive_dir, 0o775) # Maildir will throw an error if the directories are partially created # (for instance the toplevel exists but cur, new, or tmp do not) # therefore we don't create the toplevel as we did above. list_dir = os.path.join(archive_dir, mlist.fqdn_listname) mailbox = Maildir(list_dir, create=True, factory=None) lock_file = os.path.join( config.LOCK_DIR, '{0}-maildir.lock'.format(mlist.fqdn_listname)) # Lock the maildir as Maildir.add() is not threadsafe. Don't use the # context manager because it's not an error if we can't acquire the # archiver lock. We'll just log the problem and continue. # # XXX 2012-03-14 BAW: When we extend the chain/pipeline architecture # to other runners, e.g. the archive runner, it would be better to let # any TimeOutError propagate up. That would cause the message to be # re-queued and tried again later, rather than being discarded as # happens now below. lock = Lock(lock_file) try: lock.lock(timeout=timedelta(seconds=1)) # Add the message to the maildir. The return value could be used # to construct the file path if necessary. E.g. # # os.path.join(archive_dir, mlist.fqdn_listname, 'new', # message_key) mailbox.add(message) except TimeOutError: # Log the error and go on. log.error('Unable to acquire prototype archiver lock for {0}, ' 'discarding: {1}'.format( mlist.fqdn_listname, message.get('message-id', 'n/a'))) finally: lock.unlock(unconditionally=True) return None mailman-3.2.2/src/mailman/archiving/tests/0000755000175000017500000000000013445614541021602 5ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/archiving/tests/__init__.py0000644000175000017500000000000013244427337023703 0ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/archiving/tests/fake_mhonarc.py0000644000175000017500000000217313442110351024557 0ustar maxkingmaxking00000000000000# Copyright (C) 2015-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """A fake MHonArc process that reads stdin and writes stdout.""" import sys from email import message_from_string def main(): text = sys.stdin.read() msg = message_from_string(text) output_file = sys.argv[1] with open(output_file, 'w', encoding='utf-8') as fp: print(msg['message-id'], file=fp) print(msg['message-id-hash'], file=fp) if __name__ == '__main__': main() mailman-3.2.2/src/mailman/archiving/tests/test_mhonarc.py0000644000175000017500000000576013442110351024635 0ustar maxkingmaxking00000000000000# Copyright (C) 2015-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Test the MHonArc archiver.""" import os import sys import shutil import tempfile import unittest from importlib_resources import path from mailman.app.lifecycle import create_list from mailman.archiving.mhonarc import MHonArc from mailman.database.transaction import transaction from mailman.testing.helpers import ( configuration, specialized_message_from_string as mfs) from mailman.testing.layers import ConfigLayer class TestMhonarc(unittest.TestCase): """Test the MHonArc archiver.""" layer = ConfigLayer def setUp(self): # Create a fake mailing list and message object. self._msg = mfs("""\ To: test@example.com From: anne@example.com Subject: Testing the test list Message-ID: Message-ID-Hash: MS6QLWERIJLGCRF44J7USBFDELMNT2BW Tests are better than no tests but the water deserves to be swum. """) with transaction(): self._mlist = create_list('test@example.com') tempdir = tempfile.mkdtemp() self.addCleanup(shutil.rmtree, tempdir) # Here's the command to execute our fake MHonArc process. with path('mailman.archiving.tests', 'fake_mhonarc.py') as source: shutil.copy(str(source), tempdir) self._output_file = os.path.join(tempdir, 'output.txt') command = '{} {} {}'.format( sys.executable, os.path.join(tempdir, 'fake_mhonarc.py'), self._output_file) # Write an external configuration file which points the command at our # fake MHonArc process. self._cfg = os.path.join(tempdir, 'mhonarc.cfg') with open(self._cfg, 'w', encoding='utf-8') as fp: print("""\ [general] base_url: http://$hostname/archives/$fqdn_listname command: {command} """.format(command=command), file=fp) def test_mhonarc(self): # The archiver properly sends stdin to the subprocess. with configuration('archiver.mhonarc', configuration=self._cfg, enable='yes'): MHonArc().archive_message(self._mlist, self._msg) with open(self._output_file, 'r', encoding='utf-8') as fp: results = fp.read().splitlines() self.assertEqual(results[0], '') self.assertEqual(results[1], 'MS6QLWERIJLGCRF44J7USBFDELMNT2BW') mailman-3.2.2/src/mailman/archiving/tests/test_prototype.py0000644000175000017500000001566713442110351025262 0ustar maxkingmaxking00000000000000# Copyright (C) 2012-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Test the prototype archiver.""" import os import shutil import tempfile import unittest import threading from email import message_from_file from flufl.lock import Lock from mailman.app.lifecycle import create_list from mailman.archiving.prototype import Prototype from mailman.config import config from mailman.database.transaction import transaction from mailman.testing.helpers import ( LogFileMark, specialized_message_from_string as mfs) from mailman.testing.layers import ConfigLayer from mailman.utilities.email import add_message_hash class TestPrototypeArchiver(unittest.TestCase): """Test the prototype archiver.""" layer = ConfigLayer def setUp(self): # Create a fake mailing list and message object. self._msg = mfs("""\ To: test@example.com From: anne@example.com Subject: Testing the test list Message-ID: Message-ID-Hash: MS6QLWERIJLGCRF44J7USBFDELMNT2BW Tests are better than no tests but the water deserves to be swum. """) with transaction(): self._mlist = create_list('test@example.com') # Set up a temporary directory for the prototype archiver so that it's # easier to clean up. self._tempdir = tempfile.mkdtemp() self.addCleanup(shutil.rmtree, self._tempdir) config.push('prototype', """ [paths.testing] archive_dir: {} """.format(self._tempdir)) self.addCleanup(config.pop, 'prototype') # Capture the structure of a maildir. self._expected_dir_structure = set( (os.path.join(config.ARCHIVE_DIR, path) for path in ( 'prototype', os.path.join('prototype', self._mlist.fqdn_listname), os.path.join('prototype', self._mlist.fqdn_listname, 'cur'), os.path.join('prototype', self._mlist.fqdn_listname, 'new'), os.path.join('prototype', self._mlist.fqdn_listname, 'tmp'), ))) self._expected_dir_structure.add(config.ARCHIVE_DIR) def _find(self, path): all_filenames = set() for dirpath, dirnames, filenames in os.walk(path): if isinstance(dirpath, bytes): dirpath = dirpath.decode('utf-8') all_filenames.add(dirpath) for filename in filenames: new_filename = filename if isinstance(filename, bytes): new_filename = filename.decode('utf-8') all_filenames.add(os.path.join(dirpath, new_filename)) return all_filenames def test_archive_maildir_created(self): # Archiving a message to the prototype archiver should create the # expected directory structure. Prototype.archive_message(self._mlist, self._msg) all_filenames = self._find(config.ARCHIVE_DIR) # Check that the directory structure has been created and we have one # more file (the archived message) than expected directories. archived_messages = all_filenames - self._expected_dir_structure self.assertEqual(len(archived_messages), 1) self.assertTrue( archived_messages.pop().startswith( os.path.join(config.ARCHIVE_DIR, 'prototype', self._mlist.fqdn_listname, 'new'))) def test_archive_maildir_existence_does_not_raise(self): # Archiving a second message does not cause an EEXIST to be raised # when a second message is archived. new_dir = None Prototype.archive_message(self._mlist, self._msg) for directory in ('cur', 'new', 'tmp'): path = os.path.join(config.ARCHIVE_DIR, 'prototype', self._mlist.fqdn_listname, directory) if directory == 'new': new_dir = path self.assertTrue(os.path.isdir(path)) # There should be one message in the 'new' directory. self.assertEqual(len(os.listdir(new_dir)), 1) # Archive a second message. If an exception occurs, let it fail the # test. Afterward, two messages should be in the 'new' directory. del self._msg['message-id'] del self._msg['message-id-hash'] self._msg['Message-ID'] = '' add_message_hash(self._msg) Prototype.archive_message(self._mlist, self._msg) self.assertEqual(len(os.listdir(new_dir)), 2) def test_archive_lock_used(self): # Test that locking the maildir when adding works as a failure here # could mean we lose mail. lock_file = os.path.join( config.LOCK_DIR, '{0}-maildir.lock'.format( self._mlist.fqdn_listname)) with Lock(lock_file): # Acquire the archiver lock, then make sure the archiver logs the # fact that it could not acquire the lock. archive_thread = threading.Thread( target=Prototype.archive_message, args=(self._mlist, self._msg)) mark = LogFileMark('mailman.error') archive_thread.run() # Test that the archiver output the correct error. line = mark.readline() # XXX 2012-03-15 BAW: we really should remove timestamp prefixes # from the loggers when under test. self.assertTrue(line.endswith( 'Unable to acquire prototype archiver lock for {0}, ' 'discarding: {1}\n'.format( self._mlist.fqdn_listname, self._msg.get('message-id')))) # Check that the message didn't get archived. created_files = self._find(config.ARCHIVE_DIR) self.assertEqual(self._expected_dir_structure, created_files) def test_prototype_archiver_good_path(self): # Verify the good path; the message gets archived. Prototype.archive_message(self._mlist, self._msg) new_path = os.path.join( config.ARCHIVE_DIR, 'prototype', self._mlist.fqdn_listname, 'new') archived_messages = list(os.listdir(new_path)) self.assertEqual(len(archived_messages), 1) # Check that the email has been added. with open(os.path.join(new_path, archived_messages[0])) as fp: archived_message = message_from_file(fp) self.assertEqual(self._msg.as_string(), archived_message.as_string()) mailman-3.2.2/src/mailman/bin/0000755000175000017500000000000013445614541017236 5ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/bin/__init__.py0000644000175000017500000000000013244427337021337 0ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/bin/docs/0000755000175000017500000000000013445614541020166 5ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/bin/docs/master.rst0000644000175000017500000000275213244427337022223 0ustar maxkingmaxking00000000000000====================== Mailman runner control ====================== Mailman has a number of *runner subprocesses* which perform long-running tasks such as listening on an LMTP port, processing REST API requests, or processing messages in a queue directory. In normal operation, the ``mailman`` command is used to start, stop and manage the runners. This is just a wrapper around the real master watcher, which handles runner starting, stopping, exiting, and log file reopening. >>> from mailman.testing.helpers import TestableMaster Start the master in a sub-thread. >>> master = TestableMaster() >>> master.start() There should be a process id for every runner that claims to be startable. >>> from lazr.config import as_boolean >>> startable_runners = [conf for conf in config.runner_configs ... if as_boolean(conf.start)] >>> len(list(master.runner_pids)) == len(startable_runners) True Now verify that all the runners are running. :: >>> import os # This should produce no output. >>> for pid in master.runner_pids: ... os.kill(pid, 0) Stop the master process, which should also kill (and not restart) the child runner processes. >>> master.stop() None of the children are running now. >>> import errno >>> from contextlib import suppress >>> for pid in master.runner_pids: ... with suppress(ProcessLookupError): ... os.kill(pid, 0) ... print('Process did not exit:', pid) mailman-3.2.2/src/mailman/bin/mailman.py0000644000175000017500000001114313445505360021224 0ustar maxkingmaxking00000000000000# Copyright (C) 2009-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """The 'mailman' command dispatcher.""" import click from mailman.commands.cli_help import help as help_command from mailman.config import config from mailman.core.i18n import _ from mailman.core.initialize import initialize from mailman.database.transaction import transaction from mailman.interfaces.command import ICLISubCommand from mailman.utilities.modules import add_components from mailman.version import MAILMAN_VERSION_FULL from public import public class Subcommands(click.MultiCommand): # Handle dynamic listing and loading of `mailman` subcommands. def __init__(self, *args, **kws): super().__init__(*args, **kws) self._commands = {} self._loaded = False def _load(self): # Load commands lazily as commands in plugins can only be found after # the configuration file is loaded. if not self._loaded: add_components('commands', ICLISubCommand, self._commands) self._loaded = True def list_commands(self, ctx): # pragma: nocover self._load() return sorted(self._commands) def get_command(self, ctx, name): self._load() try: return self._commands[name].command except KeyError: # Returning None here signals click to report usage information # and a "No such command" error message. return None # This is here to hook command parsing into the Mailman database # transaction system. If the subcommand succeeds, the transaction is # committed, otherwise it's aborted. # See https://github.com/pallets/click/issues/1134 def invoke(self, ctx): # If given a bogus subcommand, the database won't have been # initialized so there's no transaction to commit. if config.db is not None: with transaction(): return super().invoke(ctx) return super().invoke(ctx) # pragma: missed # https://github.com/pallets/click/issues/834 # # Note that this only handles the case for the `mailman --help` output. # To handle `mailman --help` we create a custom click.Command # subclass and override this method there too. See # src/mailman/utilities/options.py def format_options(self, ctx, formatter): """Writes all the options into the formatter if they exist.""" opts = [] for param in self.get_params(ctx): rv = param.get_help_record(ctx) if rv is not None: part_a, part_b = rv opts.append((part_a, part_b.replace('\n', ' '))) if opts: with formatter.section('Options'): formatter.write_dl(opts) # Print the list of available commands. super().format_commands(ctx, formatter) def initialize_config(ctx, param, value): if not ctx.resilient_parsing: initialize(value) @click.option( '-C', '--config', 'config_file', envvar='MAILMAN_CONFIG_FILE', type=click.Path(exists=True, dir_okay=False, resolve_path=True), help=_("""\ Configuration file to use. If not given, the environment variable MAILMAN_CONFIG_FILE is consulted and used if set. If neither are given, a default configuration file is loaded."""), is_eager=True, callback=initialize_config) @click.group( cls=Subcommands, context_settings=dict(help_option_names=['-h', '--help']), invoke_without_command=True) @click.pass_context @click.version_option(MAILMAN_VERSION_FULL, message='%(version)s') @public def main(ctx, config_file): # XXX https://github.com/pallets/click/issues/303 """\ The GNU Mailman mailing list management system Copyright 1998-2018 by the Free Software Foundation, Inc. http://www.list.org """ # click handles dispatching to the subcommand via the Subcommands class. if ctx.invoked_subcommand is None: ctx.invoke(help_command) mailman-3.2.2/src/mailman/bin/master.py0000644000175000017500000005543413442110351021101 0ustar maxkingmaxking00000000000000# Copyright (C) 2001-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Master subprocess watcher.""" import os import sys import click import signal import socket import logging from datetime import timedelta from enum import Enum from flufl.lock import Lock, NotLockedError, TimeOutError from lazr.config import as_boolean from mailman.config import config from mailman.core.i18n import _ from mailman.core.initialize import initialize from mailman.core.logging import reopen from mailman.utilities.options import I18nCommand, validate_runner_spec from mailman.version import MAILMAN_VERSION_FULL from public import public DOT = '.' LOCK_LIFETIME = timedelta(days=1, hours=6) SECONDS_IN_A_DAY = 86400 SUBPROC_START_WAIT = timedelta(seconds=20) # Environment variables to forward into subprocesses. PRESERVE_ENVS = ( 'COVERAGE_PROCESS_START', 'LANG', 'LANGUAGE', 'LC_ADDRESS', 'LC_ALL', 'LC_COLLATE', 'LC_CTYPE', 'LC_IDENTIFICATION', 'LC_MEASUREMENT', 'LC_MESSAGES', 'LC_MONETARY', 'LC_NAME', 'LC_NUMERIC', 'LC_PAPER', 'LC_TELEPHONE', 'LC_TIME', 'LOCALE_ARCHIVE', 'MAILMAN_EXTRA_TESTING_CFG', 'PYTHONPATH', 'PYTHONHOME', ) @public class WatcherState(Enum): """Enum for the state of the master process watcher.""" # No lock has been acquired by any process. none = 0 # Another master watcher is running. conflict = 1 # No conflicting process exists. stale_lock = 2 # Hostname from lock file doesn't match. host_mismatch = 3 @public def master_state(lock_file=None): """Get the state of the master watcher. :param lock_file: Path to the lock file, otherwise `config.LOCK_FILE`. :type lock_file: str :return: 2-tuple of the WatcherState describing the state of the lock file, and the lock object. """ if lock_file is None: lock_file = config.LOCK_FILE # We'll never acquire the lock, so the lifetime doesn't matter. lock = Lock(lock_file) try: hostname, pid, tempfile = lock.details except NotLockedError: return WatcherState.none, lock if hostname != socket.getfqdn(): return WatcherState.host_mismatch, lock # Find out if the process exists by calling kill with a signal 0. try: os.kill(pid, 0) return WatcherState.conflict, lock except ProcessLookupError: # No matching process id. return WatcherState.stale_lock, lock def acquire_lock_1(force, lock_file=None): """Try to acquire the master lock. :param force: Flag that controls whether to force acquisition of the lock. :type force: bool :param lock_file: Path to the lock file, otherwise `config.LOCK_FILE`. :type lock_file: str :return: The master lock. :raises: `TimeOutError` if the lock could not be acquired. """ if lock_file is None: lock_file = config.LOCK_FILE lock = Lock(lock_file, LOCK_LIFETIME) try: lock.lock(timedelta(seconds=0.1)) return lock except TimeOutError: if not force: raise # Force removal of lock first. lock.disown() hostname, pid, tempfile = lock.details os.unlink(lock_file) return acquire_lock_1(force=False) def acquire_lock(force): """Acquire the master lock. :param force: Flag that controls whether to force acquisition of the lock. :type force: bool :return: The master runner lock or None if the lock couldn't be acquired. In that case, an error messages is also printed to standard error. """ try: lock = acquire_lock_1(force) return lock except TimeOutError: status, lock = master_state() if status is WatcherState.conflict: # Hostname matches and process exists. message = _("""\ The master lock could not be acquired because it appears as though another master is already running.""") elif status is WatcherState.stale_lock: # Hostname matches but the process does not exist. program = sys.argv[0] # noqa: F841 message = _("""\ The master lock could not be acquired. It appears as though there is a stale master lock. Try re-running $program with the --force flag.""") elif status is WatcherState.host_mismatch: # Hostname doesn't even match. hostname, pid, tempfile = lock.details message = _("""\ The master lock could not be acquired, because it appears as if some process on some other host may have acquired it. We can't test for stale locks across host boundaries, so you'll have to clean this up manually. Lock file: $config.LOCK_FILE Lock host: $hostname Exiting.""") else: assert status is WatcherState.none, ( 'Invalid enum value: {}'.format(status)) hostname, pid, tempfile = lock.details message = _("""\ For unknown reasons, the master lock could not be acquired. Lock file: $config.LOCK_FILE Lock host: $hostname Exiting.""") print(message, file=sys.stderr) sys.exit(1) class PIDWatcher: """A class which safely manages child process ids.""" def __init__(self): self._pids = {} def __contains__(self, pid): return pid in self._pids.keys() def __iter__(self): # Safely iterate over all the keys in the dictionary. Because # asynchronous signals are involved, the dictionary's size could # change during iteration. Iterate over a copy of the keys to avoid # that. for pid in self._pids.keys(): yield pid def add(self, pid, info): """Add process information. :param pid: The process id. The watcher must not already be tracking this process id. :type pid: int :param info: The process information. :type info: 4-tuple consisting of (runner-name, slice-number, slice-count, restart-count) """ old_info = self._pids.get(pid) assert old_info is None, ( 'Duplicate process id {0} with existing info: {1}'.format( pid, old_info)) self._pids[pid] = info def pop(self, pid): """Remove and return existing process information. :param pid: The process id. The watcher must already be tracking this process id. :type pid: int :return: The process information. :rtype: 4-tuple consisting of (runner-name, slice-number, slice-count, restart-count) :raise KeyError: if the process id is not being tracked. """ return self._pids.pop(pid) def drop(self, pid): """Remove and return existing process information. This is like `pop()` except that no `KeyError` is raised if the process id is not being tracked. :param pid: The process id. :type pid: int :return: The process information, or None if the process id is not being tracked. :rtype: 4-tuple consisting of (runner-name, slice-number, slice-count, restart-count) """ return self._pids.pop(pid, None) @public class Loop: """Main control loop class.""" def __init__(self, lock=None, restartable=None, config_file=None): self._lock = lock self._restartable = restartable self._config_file = config_file self._kids = PIDWatcher() def install_signal_handlers(self): """Install various signals handlers for control from the master.""" log = logging.getLogger('mailman.runner') # Set up our signal handlers. Also set up a SIGALRM handler to # refresh the lock once per day. The lock lifetime is 1 day + 6 hours # so this should be plenty. def sigalrm_handler(signum, frame): # noqa: E306 self._lock.refresh() signal.alarm(SECONDS_IN_A_DAY) signal.signal(signal.SIGALRM, sigalrm_handler) signal.alarm(SECONDS_IN_A_DAY) # SIGHUP tells the runners to close and reopen their log files. def sighup_handler(signum, frame): # noqa: E306 reopen() for pid in self._kids: os.kill(pid, signal.SIGHUP) log.info('Master watcher caught SIGHUP. Re-opening log files.') signal.signal(signal.SIGHUP, sighup_handler) # SIGUSR1 is used by 'mailman restart'. def sigusr1_handler(signum, frame): # noqa: E306 for pid in self._kids: os.kill(pid, signal.SIGUSR1) log.info('Master watcher caught SIGUSR1. Exiting.') signal.signal(signal.SIGUSR1, sigusr1_handler) # SIGTERM is what init will kill this process with when changing run # levels. It's also the signal 'mailman stop' uses. def sigterm_handler(signum, frame): # noqa: E306 for pid in self._kids: os.kill(pid, signal.SIGTERM) log.info('Master watcher caught SIGTERM. Exiting.') signal.signal(signal.SIGTERM, sigterm_handler) # SIGINT is what control-C gives. def sigint_handler(signum, frame): # noqa: E306 for pid in self._kids: os.kill(pid, signal.SIGINT) log.info('Master watcher caught SIGINT. Restarting.') signal.signal(signal.SIGINT, sigint_handler) def _start_runner(self, spec): """Start a runner. All arguments are passed to the process. :param spec: A runner spec, in a format acceptable to bin/runner's --runner argument, e.g. name:slice:count :type spec: string :return: The process id of the child runner. :rtype: int """ pid = os.fork() if pid: # Parent. return pid # Child. # # Set the environment variable which tells the runner that it's # running under bin/master control. This subtly changes the error # behavior of bin/runner. env = {'MAILMAN_UNDER_MASTER_CONTROL': '1'} # Craft the command line arguments for the exec() call. rswitch = '--runner=' + spec # Always pass the explicit path to the configuration file to the # sub-runners. This avoids any debate about which cfg file is used. config_file = (config.filename if self._config_file is None else self._config_file) # Wherever master lives, so too must live the runner script. exe = os.path.join(config.BIN_DIR, 'runner') # pragma: nocover # config.PYTHON, which is the absolute path to the Python interpreter, # must be given as argv[0] due to Python's library search algorithm. args = [sys.executable, sys.executable, exe, # pragma: nocover '-C', config_file, rswitch] log = logging.getLogger('mailman.runner') log.debug('starting: %s', args) # We must pass this environment variable through if it's set, # otherwise runner processes will not have the correct VAR_DIR. var_dir = os.environ.get('MAILMAN_VAR_DIR') if var_dir is not None: env['MAILMAN_VAR_DIR'] = var_dir # For the testing framework, if these environment variables are set, # pass them on to the subprocess. for envvar in PRESERVE_ENVS: if envvar in os.environ: env[envvar] = os.environ[envvar] args.append(env) os.execle(*args) # We should never get here. raise RuntimeError('os.execle() failed') def start_runners(self, runner_names=None): """Start all the configured runners. :param runners: If given, a sequence of runner names to start. If not given, this sequence is taken from the configuration file. :type runners: a sequence of strings """ if not runner_names: runner_names = [] for runner_config in config.runner_configs: # Strip off the 'runner.' prefix. assert runner_config.name.startswith('runner.'), ( 'Unexpected runner configuration section name: {}'.format( runner_config.name)) runner_names.append(runner_config.name[7:]) # For each runner we want to start, find their config section, which # will tell us the name of the class to instantiate, along with the # number of hash space slices to manage. for name in runner_names: section_name = 'runner.' + name # Let AttributeError propagate. runner_config = getattr(config, section_name) if not as_boolean(runner_config.start): continue # Find out how many runners to instantiate. This must be a power # of 2. count = int(runner_config.instances) assert (count & (count - 1)) == 0, ( 'Runner "{0}", not a power of 2: {1}'.format(name, count)) for slice_number in range(count): # runner name, slice #, # of slices, restart count info = (name, slice_number, count, 0) spec = '{0}:{1:d}:{2:d}'.format(name, slice_number, count) pid = self._start_runner(spec) log = logging.getLogger('mailman.runner') log.debug('[{0:d}] {1}'.format(pid, spec)) self._kids.add(pid, info) def _pause(self): """Sleep until a signal is received.""" # Sleep until a signal is received. This prevents the master from # exiting immediately even if there are no runners (as happens in the # test suite). signal.pause() def loop(self): """Main loop. Wait until all the runner subprocesses have exited, restarting them if necessary and configured to do so. """ log = logging.getLogger('mailman.runner') log.info('Master started') self._pause() while True: try: pid, status = os.wait() except ChildProcessError: # No children? We're done. break except InterruptedError: # pragma: nocover # If the system call got interrupted, just restart it. continue if pid not in self._kids: # pragma: nocover # This is not a runner subprocess that we own. E.g. maybe a # plugin started it. continue # Find out why the subprocess exited by getting the signal # received or exit status. if os.WIFSIGNALED(status): why = os.WTERMSIG(status) elif os.WIFEXITED(status): why = os.WEXITSTATUS(status) else: why = None # We'll restart the subprocess if it exited with a SIGUSR1 or # because of a failure (i.e. no exit signal), and the no-restart # command line switch was not given. This lets us better handle # runaway restarts (e.g. if the subprocess had a syntax error!) rname, slice_number, count, restarts = self._kids.pop(pid) config_name = 'runner.' + rname restart = False if why == signal.SIGUSR1 and self._restartable: restart = True # Have we hit the maximum number of restarts? restarts += 1 max_restarts = int(getattr(config, config_name).max_restarts) if restarts > max_restarts: restart = False # Are we permanently non-restartable? log.debug("""\ Master detected subprocess exit (pid: {0:d}, why: {1}, class: {2}, slice: {3:d}/{4:d}) {5}""".format( pid, why, rname, slice_number + 1, count, ('[restarting]' if restart else ''))) # See if we've reached the maximum number of allowable restarts. if restarts > max_restarts: log.info("""\ Runner {0} reached maximum restart limit of {1:d}, not restarting.""", rname, max_restarts) # Now perhaps restart the process unless it exited with a # SIGTERM or we aren't restarting. if restart: spec = '{0}:{1:d}:{2:d}'.format(rname, slice_number, count) new_pid = self._start_runner(spec) new_info = (rname, slice_number, count, restarts) self._kids.add(new_pid, new_info) log.info('Master stopped') def cleanup(self): """Ensure that all children have exited.""" log = logging.getLogger('mailman.runner') # Send SIGTERMs to all the child processes and wait for them all to # exit. for pid in self._kids: try: os.kill(pid, signal.SIGTERM) except ProcessLookupError: # pragma: nocover # The child has already exited. log.info('ESRCH on pid: %d', pid) except OSError: # pragma: nocover # XXX I'm not so sure about this. It preserves the semantics # before conversion to PEP 3151 exceptions. But is it right? pass # Wait for all the children to go away. while self._kids: try: pid, status = os.wait() self._kids.drop(pid) except ChildProcessError: break except InterruptedError: # pragma: nocover continue @click.command( cls=I18nCommand, context_settings=dict(help_option_names=['-h', '--help']), help=_("""\ Master subprocess watcher. Start and watch the configured runners, ensuring that they stay alive and kicking. Each runner is forked and exec'd in turn, with the master waiting on their process ids. When it detects a child runner has exited, it may restart it. The runners respond to SIGINT, SIGTERM, SIGUSR1 and SIGHUP. SIGINT, SIGTERM and SIGUSR1 all cause a runner to exit cleanly. The master will restart runners that have exited due to a SIGUSR1 or some kind of other exit condition (say because of an uncaught exception). SIGHUP causes the master and the runners to close their log files, and reopen then upon the next printed message. The master also responds to SIGINT, SIGTERM, SIGUSR1 and SIGHUP, which it simply passes on to the runners. Note that the master will close and reopen its own log files on receipt of a SIGHUP. The master also leaves its own process id in the file specified in the configuration file but you normally don't need to use this PID directly.""")) @click.option( '-C', '--config', 'config_file', envvar='MAILMAN_CONFIG_FILE', type=click.Path(exists=True, dir_okay=False, resolve_path=True), help=_("""\ Configuration file to use. If not given, the environment variable MAILMAN_CONFIG_FILE is consulted and used if set. If neither are given, a default configuration file is loaded.""")) @click.option( '--no-restart', '-n', 'restartable', is_flag=True, default=True, help=_("""\ Don't restart the runners when they exit because of an error or a SIGUSR1. Use this only for debugging.""")) @click.option( '--force', '-f', is_flag=True, default=False, help=_("""\ If the master watcher finds an existing master lock, it will normally exit with an error message. With this option,the master will perform an extra level of checking. If a process matching the host/pid described in the lock file is running, the master will still exit, requiring you to manually clean up the lock. But if no matching process is found, the master will remove the apparently stale lock and make another attempt to claim the master lock.""")) @click.option( '--runners', '-r', metavar='runner[:slice:range]', callback=validate_runner_spec, default=None, multiple=True, help=_("""\ Override the default set of runners that the master will invoke, which is typically defined in the configuration file. Multiple -r options may be given. The values for -r are passed straight through to bin/runner.""")) @click.option( '-v', '--verbose', is_flag=True, default=False, help=_('Display more debugging information to the log file.')) @click.version_option(MAILMAN_VERSION_FULL) @public def main(config_file, restartable, force, runners, verbose): # XXX https://github.com/pallets/click/issues/303 """Master subprocess watcher. Start and watch the configured runners and ensure that they stay alive and kicking. Each runner is forked and exec'd in turn, with the master waiting on their process ids. When it detects a child runner has exited, it may restart it. The runners respond to SIGINT, SIGTERM, SIGUSR1 and SIGHUP. SIGINT, SIGTERM and SIGUSR1 all cause a runner to exit cleanly. The master will restart runners that have exited due to a SIGUSR1 or some kind of other exit condition (say because of an uncaught exception). SIGHUP causes the master and the runners to close their log files, and reopen then upon the next printed message. The master also responds to SIGINT, SIGTERM, SIGUSR1 and SIGHUP, which it simply passes on to the runners. Note that the master will close and reopen its own log files on receipt of a SIGHUP. The master also leaves its own process id in the file `data/master.pid` but you normally don't need to use this pid directly. """ initialize(config_file, verbose) # Acquire the master lock, exiting if we can't. We'll let the caller # handle any clean up or lock breaking. No `with` statement here because # Lock's constructor doesn't support a timeout. lock = acquire_lock(force) try: with open(config.PID_FILE, 'w') as fp: print(os.getpid(), file=fp) loop = Loop(lock, restartable, config.filename) loop.install_signal_handlers() try: loop.start_runners(runners) loop.loop() finally: loop.cleanup() os.remove(config.PID_FILE) finally: lock.unlock() mailman-3.2.2/src/mailman/bin/runner.py0000644000175000017500000001615713442110351021116 0ustar maxkingmaxking00000000000000# Copyright (C) 2001-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """The runner process.""" import os import sys import click import signal import logging import traceback from mailman.config import config from mailman.core.i18n import _ from mailman.core.initialize import initialize from mailman.utilities.modules import find_name from mailman.utilities.options import I18nCommand, validate_runner_spec from mailman.version import MAILMAN_VERSION_FULL from public import public log = None # Enable coverage if run under the appropriate test suite. if os.environ.get('COVERAGE_PROCESS_START') is not None: import coverage coverage.process_startup() def make_runner(name, slice, range, once=False): # The runner name must be defined in the configuration. Only runner short # names are supported. runner_config = getattr(config, 'runner.' + name, None) if runner_config is None: print(_('Undefined runner name: $name'), file=sys.stderr) # Exit with SIGTERM exit code so the master won't try to restart us. sys.exit(signal.SIGTERM) class_path = runner_config['class'] try: runner_class = find_name(class_path) except ImportError: if os.environ.get('MAILMAN_UNDER_MASTER_CONTROL') is not None: print(_('Cannot import runner module: $class_path'), file=sys.stderr) traceback.print_exc() sys.exit(signal.SIGTERM) else: raise if once: # Subclass to hack in the setting of the stop flag in _do_periodic() class Once(runner_class): def _do_periodic(self): self.stop() return Once(name, slice) return runner_class(name, slice) @click.command( cls=I18nCommand, context_settings=dict(help_option_names=['-h', '--help']), help=_("""\ Start a runner. The runner named on the command line is started, and it can either run through its main loop once (for those runners that support this) or continuously. The latter is how the master runner starts all its subprocesses. -r is required unless -l or -h is given, and its argument must be one of the names displayed by the -l switch. Normally, this script should be started from `mailman start`. Running it separately or with -o is generally useful only for debugging. When run this way, the environment variable $MAILMAN_UNDER_MASTER_CONTROL will be set which subtly changes some error handling behavior. """)) @click.option( '-C', '--config', 'config_file', envvar='MAILMAN_CONFIG_FILE', type=click.Path(exists=True, dir_okay=False, resolve_path=True), help=_("""\ Configuration file to use. If not given, the environment variable MAILMAN_CONFIG_FILE is consulted and used if set. If neither are given, a default configuration file is loaded.""")) @click.option( '-l', '--list', 'list_runners', is_flag=True, is_eager=True, default=False, help=_('List the available runner names and exit.')) @click.option( '-o', '--once', is_flag=True, default=False, help=_("""\ Run the named runner exactly once through its main loop. Otherwise, the runner runs indefinitely until the process receives a signal. This is not compatible with runners that cannot be run once.""")) @click.option( '-r', '--runner', 'runner_spec', metavar='runner[:slice:range]', callback=validate_runner_spec, default=None, help=_("""\ Start the named runner, which must be one of the strings returned by the -l option. For runners that manage a queue directory, optional `slice:range` if given is used to assign multiple runner processes to that queue. range is the total number of runners for the queue while slice is the number of this runner from [0..range). For runners that do not manage a queue, slice and range are ignored. When using the `slice:range` form, you must ensure that each runner for the queue is given the same range value. If `slice:runner` is not given, then 1:1 is used. """)) @click.option( '-v', '--verbose', is_flag=True, default=False, help=_('Display more debugging information to the log file.')) @click.version_option(MAILMAN_VERSION_FULL) @click.pass_context @public def main(ctx, config_file, verbose, list_runners, once, runner_spec): # XXX https://github.com/pallets/click/issues/303 """Start a runner. The runner named on the command line is started, and it can either run through its main loop once (for those runners that support this) or continuously. The latter is how the master runner starts all its subprocesses. -r is required unless -l or -h is given, and its argument must be one of the names displayed by the -l switch. Normally, this script should be started from 'mailman start'. Running it separately or with -o is generally useful only for debugging. When run this way, the environment variable $MAILMAN_UNDER_MASTER_CONTROL will be set which subtly changes some error handling behavior. """ global log if runner_spec is None and not list_runners: ctx.fail(_('No runner name given.')) # Initialize the system. Honor the -C flag if given. initialize(config_file, verbose) log = logging.getLogger('mailman.runner') if verbose: console = logging.StreamHandler(sys.stderr) formatter = logging.Formatter(config.logging.root.format, config.logging.root.datefmt) console.setFormatter(formatter) logging.getLogger().addHandler(console) logging.getLogger().setLevel(logging.DEBUG) if list_runners: descriptions = {} for section in config.runner_configs: ignore, dot, shortname = section.name.rpartition('.') ignore, dot, classname = getattr(section, 'class').rpartition('.') descriptions[shortname] = classname longest = max(len(name) for name in descriptions) for shortname in sorted(descriptions): classname = descriptions[shortname] spaces = longest - len(shortname) name = (' ' * spaces) + shortname # noqa: F841 print(_('$name runs $classname')) sys.exit(0) runner = make_runner(*runner_spec, once=once) runner.set_signals() # Now start up the main loop log.info('{} runner started.'.format(runner.name)) runner.run() log.info('{} runner exiting.'.format(runner.name)) sys.exit(runner.status) mailman-3.2.2/src/mailman/bin/tests/0000755000175000017500000000000013445614541020400 5ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/bin/tests/__init__.py0000644000175000017500000000000013244427337022501 0ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/bin/tests/test_mailman.py0000644000175000017500000001147313442110351023420 0ustar maxkingmaxking00000000000000# Copyright (C) 2015-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Test mailman command utilities.""" import unittest from click.testing import CliRunner from datetime import timedelta from importlib_resources import path from mailman.app.lifecycle import create_list from mailman.bin.mailman import main from mailman.config import config from mailman.database.transaction import transaction from mailman.interfaces.command import ICLISubCommand from mailman.testing.layers import ConfigLayer from mailman.utilities.datetime import now from mailman.utilities.modules import add_components from unittest.mock import patch class TestMailmanCommand(unittest.TestCase): layer = ConfigLayer def setUp(self): self._command = CliRunner() def test_mailman_command_config(self): with path('mailman.testing', 'testing.cfg') as config_path: with patch('mailman.bin.mailman.initialize') as init: self._command.invoke(main, ('-C', str(config_path), 'info')) init.assert_called_once_with(str(config_path)) def test_mailman_command_no_config(self): with patch('mailman.bin.mailman.initialize') as init: self._command.invoke(main, ('info',)) init.assert_called_once_with(None) @patch('mailman.bin.mailman.initialize') def test_mailman_command_without_subcommand_prints_help(self, mock): # Issue #137: Running `mailman` without a subcommand raises an # AttributeError. result = self._command.invoke(main, []) lines = result.output.splitlines() # "main" instead of "mailman" because of the way the click runner # works. It does actually show the correct program when run from the # command line. self.assertEqual(lines[0], 'Usage: main [OPTIONS] COMMAND [ARGS]...') # The help output includes a list of subcommands, in sorted order. commands = {} add_components('commands', ICLISubCommand, commands) help_commands = list( line.split()[0].strip() for line in lines[-len(commands):] ) self.assertEqual(sorted(commands), help_commands) @patch('mailman.bin.mailman.initialize') def test_mailman_command_with_bad_subcommand_prints_help(self, mock): # Issue #137: Running `mailman` without a subcommand raises an # AttributeError. result = self._command.invoke(main, ('not-a-subcommand',)) lines = result.output.splitlines() # "main" instead of "mailman" because of the way the click runner # works. It does actually show the correct program when run from the # command line. self.assertEqual(lines[0], 'Usage: main [OPTIONS] COMMAND [ARGS]...') @patch('mailman.bin.mailman.initialize') def test_transaction_commit_after_successful_subcommand(self, mock): # Issue #223: Subcommands which change the database need to commit or # abort the transaction. with transaction(): mlist = create_list('ant@example.com') mlist.volume = 5 mlist.next_digest_number = 3 mlist.digest_last_sent_at = now() - timedelta(days=60) self._command.invoke(main, ('digests', '-b', '-l', 'ant@example.com')) # Clear the current transaction to force a database reload. config.db.abort() self.assertEqual(mlist.volume, 6) self.assertEqual(mlist.next_digest_number, 1) @patch('mailman.bin.mailman.initialize') @patch('mailman.commands.cli_digests.maybe_send_digest_now', side_effect=RuntimeError) def test_transaction_abort_after_failing_subcommand(self, mock1, mock2): with transaction(): mlist = create_list('ant@example.com') mlist.volume = 5 mlist.next_digest_number = 3 mlist.digest_last_sent_at = now() - timedelta(days=60) self._command.invoke( main, ('digests', '-b', '-l', 'ant@example.com', '--send')) # Clear the current transaction to force a database reload. config.db.abort() # The volume and number haven't changed. self.assertEqual(mlist.volume, 5) self.assertEqual(mlist.next_digest_number, 3) mailman-3.2.2/src/mailman/bin/tests/test_master.py0000644000175000017500000001116413442110351023272 0ustar maxkingmaxking00000000000000# Copyright (C) 2010-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Test master watcher utilities.""" import os import tempfile import unittest from click.testing import CliRunner from contextlib import ExitStack, suppress from datetime import timedelta from flufl.lock import Lock, TimeOutError from importlib_resources import path from io import StringIO from mailman.bin import master from mailman.config import config from mailman.testing.layers import ConfigLayer from unittest.mock import patch class FakeLock: details = ('host.example.com', 9999, '/tmp/whatever') def unlock(self): pass class TestMaster(unittest.TestCase): layer = ConfigLayer def setUp(self): fd, self.lock_file = tempfile.mkstemp() os.close(fd) # The lock file should not exist before we try to acquire it. os.remove(self.lock_file) def tearDown(self): # Unlocking removes the lock file, but just to be safe (i.e. in case # of errors). with suppress(FileNotFoundError): os.remove(self.lock_file) def test_acquire_lock_1(self): lock = master.acquire_lock_1(False, self.lock_file) is_locked = lock.is_locked lock.unlock() self.assertTrue(is_locked) def test_master_state(self): my_lock = Lock(self.lock_file) # Mailman is not running. state, lock = master.master_state(self.lock_file) self.assertEqual(state, master.WatcherState.none) # Acquire the lock as if another process had already started the # master. Use a timeout to avoid this test deadlocking. my_lock.lock(timedelta(seconds=60)) try: state, lock = master.master_state(self.lock_file) finally: my_lock.unlock() self.assertEqual(state, master.WatcherState.conflict) def test_acquire_lock_timeout_reason_unknown(self): stderr = StringIO() with ExitStack() as resources: resources.enter_context(patch( 'mailman.bin.master.acquire_lock_1', side_effect=TimeOutError)) resources.enter_context(patch( 'mailman.bin.master.master_state', return_value=(master.WatcherState.none, FakeLock()))) resources.enter_context(patch( 'mailman.bin.master.sys.stderr', stderr)) with self.assertRaises(SystemExit) as cm: master.acquire_lock(False) self.assertEqual(cm.exception.code, 1) self.assertEqual(stderr.getvalue(), """\ For unknown reasons, the master lock could not be acquired. Lock file: {} Lock host: host.example.com Exiting. """.format(config.LOCK_FILE)) def test_main_cli(self): command = CliRunner() fake_lock = FakeLock() with ExitStack() as resources: config_file = str(resources.enter_context( path('mailman.testing', 'testing.cfg'))) init_mock = resources.enter_context(patch( 'mailman.bin.master.initialize')) lock_mock = resources.enter_context(patch( 'mailman.bin.master.acquire_lock', return_value=fake_lock)) start_mock = resources.enter_context(patch.object( master.Loop, 'start_runners')) loop_mock = resources.enter_context(patch.object( master.Loop, 'loop')) command.invoke( master.main, ('-C', config_file, '--no-restart', '--force', '-r', 'in:1:1', '--verbose')) # We got initialized with the custom configuration file and the # verbose flag. init_mock.assert_called_once_with(config_file, True) # We returned a lock that was force-acquired. lock_mock.assert_called_once_with(True) # We created a non-restartable loop. start_mock.assert_called_once_with([('in', 1, 1)]) loop_mock.assert_called_once_with() mailman-3.2.2/src/mailman/chains/0000755000175000017500000000000013445614541017733 5ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/chains/__init__.py0000644000175000017500000000000013244427337022034 0ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/chains/accept.py0000644000175000017500000000362313442110351021533 0ustar maxkingmaxking00000000000000# Copyright (C) 2007-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """The terminal 'accept' chain.""" import logging from mailman.chains.base import TerminalChainBase from mailman.config import config from mailman.core.i18n import _ from mailman.interfaces.chain import AcceptEvent from public import public from zope.event import notify log = logging.getLogger('mailman.vette') SEMISPACE = '; ' @public class AcceptChain(TerminalChainBase): """Accept the message for posting.""" name = 'accept' description = _('Accept a message.') def _process(self, mlist, msg, msgdata): """See `TerminalChainBase`.""" # Start by decorating the message with a header that contains a list # of all the rules that matched. These metadata could be None or an # empty list. rule_hits = msgdata.get('rule_hits') if rule_hits: msg['X-Mailman-Rule-Hits'] = SEMISPACE.join(rule_hits) rule_misses = msgdata.get('rule_misses') if rule_misses: msg['X-Mailman-Rule-Misses'] = SEMISPACE.join(rule_misses) config.switchboards['pipeline'].enqueue(msg, msgdata) log.info('ACCEPT: %s', msg.get('message-id', 'n/a')) notify(AcceptEvent(mlist, msg, msgdata, self)) mailman-3.2.2/src/mailman/chains/base.py0000644000175000017500000001143513442110351021206 0ustar maxkingmaxking00000000000000# Copyright (C) 2008-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Base class for terminal chains.""" from mailman.config import config from mailman.interfaces.chain import ( IChain, IChainIterator, IChainLink, IMutableChain, LinkAction) from mailman.interfaces.rules import IRule from mailman.utilities.modules import abstract_component from public import public from zope.interface import implementer @public @implementer(IChainLink) class Link: """A chain link.""" def __init__(self, rule, action=None, chain=None, function=None): self.rule = (rule if IRule.providedBy(rule) else config.rules[rule]) self.action = (LinkAction.defer if action is None else action) self.chain = (chain if chain is None or IChain.providedBy(chain) else config.chains[chain]) self.function = function def __repr__(self): message = '. """The default built-in starting chain.""" import logging from mailman.chains.base import Link from mailman.core.i18n import _ from mailman.interfaces.chain import IChain, LinkAction from public import public from zope.interface import implementer log = logging.getLogger('mailman.vette') @public @implementer(IChain) class BuiltInChain: """Default built-in chain.""" name = 'default-posting-chain' description = _('The built-in moderation chain.') _link_descriptions = ( # First check DMARC. For a reject or discard, the rule hits and we # jump to the moderation chain to do the action. Otherwise, the rule # misses buts sets msgdata['dmarc'] for the handler. ('dmarc-mitigation', LinkAction.jump, 'dmarc'), # Discard emails with no valid senders. ('no-senders', LinkAction.jump, 'discard'), ('approved', LinkAction.jump, 'accept'), ('emergency', LinkAction.jump, 'hold'), ('loop', LinkAction.jump, 'discard'), # Discard emails from banned addresses. ('banned-address', LinkAction.jump, 'discard'), # Determine whether the member or nonmember has an action shortcut. ('member-moderation', LinkAction.jump, 'moderation'), # Take a detour through the header matching chain. ('truth', LinkAction.detour, 'header-match'), # Check for nonmember moderation. ('nonmember-moderation', LinkAction.jump, 'moderation'), # Do all of the following before deciding whether to hold the message. ('administrivia', LinkAction.defer, None), ('implicit-dest', LinkAction.defer, None), ('max-recipients', LinkAction.defer, None), ('max-size', LinkAction.defer, None), ('news-moderation', LinkAction.defer, None), ('no-subject', LinkAction.defer, None), ('suspicious-header', LinkAction.defer, None), # Now if any of the above hit, jump to the hold chain. ('any', LinkAction.jump, 'hold'), # Finally, the builtin chain jumps to acceptance. ('truth', LinkAction.jump, 'accept'), ) def __init__(self): self._cached_links = None def get_links(self, mlist, msg, msgdata): """See `IChain`.""" if self._cached_links is None: self._cached_links = links = [] for rule, action, chain in self._link_descriptions: links.append(Link(rule, action, chain)) return iter(self._cached_links) mailman-3.2.2/src/mailman/chains/discard.py0000644000175000017500000000277413442110351021713 0ustar maxkingmaxking00000000000000# Copyright (C) 2007-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """The terminal 'discard' chain.""" import logging from mailman.chains.base import TerminalChainBase from mailman.core.i18n import _ from mailman.interfaces.chain import DiscardEvent from public import public from zope.event import notify log = logging.getLogger('mailman.vette') @public class DiscardChain(TerminalChainBase): """Discard a message.""" name = 'discard' description = _('Discard a message and stop processing.') def _process(self, mlist, msg, msgdata): """See `TerminalChainBase`. This writes a log message, fires a Zope event and then throws the message away. """ log.info('DISCARD: %s', msg.get('message-id', 'n/a')) notify(DiscardEvent(mlist, msg, msgdata, self)) # Nothing more needs to happen. mailman-3.2.2/src/mailman/chains/dmarc.py0000644000175000017500000000262713442110351021365 0ustar maxkingmaxking00000000000000# Copyright (C) 2017-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """DMARC mitigation chain.""" from mailman.chains.base import JumpChainBase from mailman.core.i18n import _ from public import public @public class DMARCMitigationChain(JumpChainBase): """Perform DMARC mitigation.""" name = 'dmarc' description = _('Process DMARC reject or discard mitigations') def jump_to(self, mlist, msg, msgdata): # Which action should be taken? jump_chain = msgdata['dmarc_action'] assert jump_chain in ('discard', 'reject'), ( '{}: Invalid DMARC action: {} for sender: {}'.format( mlist.list_id, jump_chain, msgdata.get('moderation_sender', '(unknown)'))) return jump_chain mailman-3.2.2/src/mailman/chains/docs/0000755000175000017500000000000013445614541020663 5ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/chains/docs/__init__.py0000644000175000017500000000000013244427337022764 0ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/chains/docs/chains.rst0000644000175000017500000000006513244427337022665 0ustar maxkingmaxking00000000000000====== Chains ====== .. toctree:: :glob: ./* mailman-3.2.2/src/mailman/chains/docs/moderation.rst0000644000175000017500000001631313421245331023551 0ustar maxkingmaxking00000000000000========== Moderation ========== Posts by members and nonmembers are subject to moderation checks during incoming processing. Different situations can cause such posts to be held for moderator approval. >>> mlist = create_list('test@example.com') Members and nonmembers have a *moderation action* which can shortcut the normal moderation checks. The built-in chain does just a few checks first, such as seeing if the message has a matching `Approved:` header, or if the emergency flag has been set on the mailing list, or whether a mail loop has been detected. Mailing lists have a default moderation action, one for members and another for nonmembers. If a member's moderation action is ``None``, then the member moderation check falls back to the appropriate list default. A moderation action of `defer` means that no explicit moderation check is performed and the rest of the rule chain processing proceeds as normal. But it is also common for first-time posters to have a `hold` action, meaning that their messages are held for moderator approval for a while. Nonmembers almost always have a `hold` action, though some mailing lists may choose to set this default action to `discard`, meaning their posts would be immediately thrown away. Member moderation ================= Posts by list members are moderated if the member's moderation action is not deferred. The default setting for the moderation action of new members is determined by the mailing list's settings. By default, a mailing list is not set to moderate new member postings. >>> print(mlist.default_member_action) Action.defer In order to find out whether the message is held or accepted, we can subscribe to internal events that are triggered on each case. >>> from mailman.interfaces.chain import ChainEvent >>> def on_chain(event): ... if isinstance(event, ChainEvent): ... print(event) ... print(event.chain) ... print('Subject:', event.msg['subject']) ... print('Hits:') ... for hit in event.msgdata.get('rule_hits', []): ... print(' ', hit) ... print('Misses:') ... for miss in event.msgdata.get('rule_misses', []): ... print(' ', miss) Anne is a list member with moderation action of ``None`` so that moderation will fall back to the mailing list's ``default_member_action``. >>> from mailman.testing.helpers import subscribe >>> member = subscribe(mlist, 'Anne', email='anne@example.com') >>> member on test@example.com as MemberRole.member> >>> print(member.moderation_action) None Anne's post to the mailing list runs through the incoming runner's default built-in chain. No rules hit and so the message is accepted. :: >>> msg = message_from_string("""\ ... From: anne@example.com ... To: test@example.com ... Subject: aardvark ... ... This is a test. ... """) >>> from mailman.core.chains import process >>> from mailman.testing.helpers import event_subscribers >>> with event_subscribers(on_chain): ... process(mlist, msg, {}, 'default-posting-chain') Subject: aardvark Hits: Misses: dmarc-mitigation no-senders approved emergency loop banned-address member-moderation nonmember-moderation administrivia implicit-dest max-recipients max-size news-moderation no-subject suspicious-header However, when Anne's moderation action is set to `hold`, her post is held for moderator approval. :: >>> from mailman.interfaces.action import Action >>> member.moderation_action = Action.hold >>> msg = message_from_string("""\ ... From: anne@example.com ... To: test@example.com ... Subject: badger ... ... This is a test. ... """) >>> with event_subscribers(on_chain): ... process(mlist, msg, {}, 'default-posting-chain') Subject: badger Hits: member-moderation Misses: dmarc-mitigation no-senders approved emergency loop banned-address Anne's moderation action can also be set to `discard`... :: >>> member.moderation_action = Action.discard >>> msg = message_from_string("""\ ... From: anne@example.com ... To: test@example.com ... Subject: cougar ... ... This is a test. ... """) >>> with event_subscribers(on_chain): ... process(mlist, msg, {}, 'default-posting-chain') Subject: cougar Hits: member-moderation Misses: dmarc-mitigation no-senders approved emergency loop banned-address ... or `reject`. >>> member.moderation_action = Action.reject >>> msg = message_from_string("""\ ... From: anne@example.com ... To: test@example.com ... Subject: dingo ... ... This is a test. ... """) >>> with event_subscribers(on_chain): ... process(mlist, msg, {}, 'default-posting-chain') Subject: dingo Hits: member-moderation Misses: dmarc-mitigation no-senders approved emergency loop banned-address Nonmembers ========== Registered nonmembers are handled very similarly to members, except that a different list default setting is used when moderating nonmemberds. This is how the incoming runner adds sender addresses as nonmembers. >>> from zope.component import getUtility >>> from mailman.interfaces.usermanager import IUserManager >>> user_manager = getUtility(IUserManager) >>> address = user_manager.create_address('bart@example.com') >>> address When the moderation rule runs on a message from this sender, this address will be registered as a nonmember of the mailing list, and it will be held for moderator approval. :: >>> msg = message_from_string("""\ ... From: bart@example.com ... To: test@example.com ... Subject: elephant ... ... """) >>> with event_subscribers(on_chain): ... process(mlist, msg, {}, 'default-posting-chain') Subject: elephant Hits: nonmember-moderation Misses: dmarc-mitigation no-senders approved emergency loop banned-address member-moderation >>> nonmember = mlist.nonmembers.get_member('bart@example.com') >>> nonmember When a nonmember's default moderation action is ``None``, the rule will use the mailing list's ``default_nonmember_action``. >>> print(nonmember.moderation_action) None >>> print(mlist.default_nonmember_action) Action.hold mailman-3.2.2/src/mailman/chains/headers.py0000644000175000017500000001654313442110351021714 0ustar maxkingmaxking00000000000000# Copyright (C) 2007-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """The header-matching chain.""" import re import logging from email.header import Header from itertools import count from mailman.chains.base import Chain, Link from mailman.config import config from mailman.core.i18n import _ from mailman.interfaces.chain import LinkAction from mailman.interfaces.rules import IRule from public import public from zope.interface import implementer log = logging.getLogger('mailman.error') _RULE_COUNTER = count(1) def _make_rule_name(suffix): # suffix may be None, since it comes from the 'name' parameter given in # the HeaderMatchRule constructor. if suffix is None: suffix = '{0:02}'.format(next(_RULE_COUNTER)) return 'header-match-{}'.format(suffix) def make_link(header, pattern, chain=None, suffix=None): """Create a Link object. The link action is to defer by default, since at the end of all the header checks, we'll jump to the chain defined in the configuration file, should any of them have matched. However, it is possible to create a link which jumps to a specific chain. :param header: The email header name to check, e.g. X-Spam. :type header: string :param pattern: A regular expression for matching the header value. :type pattern: string :param chain: When given, this is the name of the chain to jump to if the pattern matches the header. :type chain: string :param suffix: An optional name suffix for the rule. :type suffix: string :return: The link representing this rule check. :rtype: `ILink` """ rule_name = _make_rule_name(suffix) if rule_name in config.rules: rule = config.rules[rule_name] else: rule = HeaderMatchRule(header, pattern, suffix) if chain is None: return Link(rule) return Link(rule, LinkAction.jump, chain) @implementer(IRule) class HeaderMatchRule: """Header matching rule used by header-match chain.""" def __init__(self, header, pattern, suffix=None): self.header = header self.pattern = pattern self.name = _make_rule_name(suffix) self.description = '{}: {}'.format(header, pattern) # XXX I think we should do better here, somehow recording that a # particular header matched a particular pattern, but that gets ugly # with RFC 2822 headers. It also doesn't match well with the rule # name concept. For now, we just record the rather useless numeric # rule name. I suppose we could do the better hit recording in the # check() method, and set self.record = False. self.record = True # Register this rule so that other parts of the system can query it. assert self.name not in config.rules, ( 'Duplicate HeaderMatchRule: {} [{}: {}]'.format( self.name, self.header, self.pattern)) config.rules[self.name] = self def check(self, mlist, msg, msgdata): """See `IRule`.""" # Collect all the headers in all subparts. headers = [] for p in msg.walk(): headers.extend(p.get_all(self.header, [])) for value in headers: if isinstance(value, Header): value = value.encode() try: mo = re.search(self.pattern, value, re.IGNORECASE) except re.error as error: log.error( "Invalid regexp '{}' in header_matches for {}: {}".format( self.pattern, mlist.list_id, error.msg)) return False else: if mo: msgdata['moderation_sender'] = msg.sender with _.defer_translation(): # This will be translated at the point of use. msgdata.setdefault('moderation_reasons', []).append( (_('Header "{}" matched a header rule'), str(value))) return True return False @public class HeaderMatchChain(Chain): """Default header matching chain. This could be extended by header match rules in the database. """ def __init__(self): super().__init__( 'header-match', _('The built-in header matching chain')) # This chain will dynamically calculate the links from the # configuration file, the database, and any explicitly added header # checks (via the .extend() method). self._extended_links = [] def extend(self, header, pattern): """Extend the existing header matches. :param header: The case-insensitive header field name. :param pattern: The pattern to match the header's value again. The match is not anchored and is done case-insensitively. """ self._extended_links.append(make_link(header, pattern)) def flush(self): """See `IMutableChain`.""" # Remove all dynamically created rules. Use the keys so we can mutate # the dictionary inside the loop. for rule_name in list(config.rules): if rule_name.startswith('header-match-'): del config.rules[rule_name] self._extended_links = [] def get_links(self, mlist, msg, msgdata): """See `IChain`.""" # First return all the configuration file links. for index, line in enumerate( config.antispam.header_checks.splitlines()): if len(line.strip()) == 0: continue parts = line.split(':', 1) if len(parts) != 2: log.error('Configuration error: [antispam]header_checks ' 'contains bogus line: {}'.format(line)) continue rule_name = 'config-{}'.format(index) yield make_link(parts[0], parts[1].lstrip(), suffix=rule_name) # Then return all the explicitly added links. yield from self._extended_links # If any of the above rules matched, they will have deferred their # action until now, so jump to the chain defined in the configuration # file. For security considerations, this takes precedence over # list-specific matches. yield Link('any', LinkAction.jump, config.antispam.jump_chain) # Then return all the list-specific header matches. for index, entry in enumerate(mlist.header_matches): # Jump to the default antispam chain if the entry chain is None. chain = (config.antispam.jump_chain if entry.chain is None else entry.chain) rule_name = '{}-{}'.format(mlist.list_id, index) yield make_link(entry.header, entry.pattern, chain, rule_name) mailman-3.2.2/src/mailman/chains/hold.py0000644000175000017500000002715313442110351021226 0ustar maxkingmaxking00000000000000# Copyright (C) 2007-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """The terminal 'hold' chain.""" import logging from email.mime.message import MIMEMessage from email.mime.text import MIMEText from email.utils import formatdate, make_msgid from mailman.app.moderator import hold_message from mailman.app.replybot import can_acknowledge from mailman.chains.base import TerminalChainBase from mailman.config import config from mailman.core.i18n import _, format_reasons from mailman.email.message import UserNotification from mailman.interfaces.autorespond import IAutoResponseSet, Response from mailman.interfaces.chain import HoldEvent from mailman.interfaces.languages import ILanguageManager from mailman.interfaces.pending import IPendable, IPendings from mailman.interfaces.template import ITemplateLoader from mailman.interfaces.usermanager import IUserManager from mailman.utilities.string import expand, oneline, wrap from public import public from zope.component import getUtility from zope.event import notify from zope.interface import implementer SEMISPACE = '; ' SPACE = ' ' NL = '\n' log = logging.getLogger('mailman.vette') @implementer(IPendable) class HeldMessagePendable(dict): PEND_TYPE = 'held message' def _compose_reasons(msgdata, column=66): # Rules can add reasons to the metadata. reasons = msgdata.get('moderation_reasons', [_('N/A')]) return NL.join( [(SPACE * 4) + wrap(reason, column=column) for reason in format_reasons(reasons)]) def autorespond_to_sender(mlist, sender, language=None): """Should Mailman automatically respond to this sender? :param mlist: The mailing list. :type mlist: `IMailingList`. :param sender: The sender's email address. :type sender: string :param language: Optional language. :type language: `ILanguage` or None :return: True if an automatic response should be sent, otherwise False. If an automatic response is not sent, a message is sent indicating that, er no more will be sent today. :rtype: bool """ if language is None: language = mlist.preferred_language max_autoresponses_per_day = int(config.mta.max_autoresponses_per_day) if max_autoresponses_per_day == 0: # Unlimited. return True # Get an IAddress from an email address. user_manager = getUtility(IUserManager) address = user_manager.get_address(sender) if address is None: address = user_manager.create_address(sender) response_set = IAutoResponseSet(mlist) todays_count = response_set.todays_count(address, Response.hold) if todays_count < max_autoresponses_per_day: # This person has not reached their automatic response limit, so it's # okay to send a response. response_set.response_sent(address, Response.hold) return True elif todays_count == max_autoresponses_per_day: # The last one we sent was the last one we should send today. Instead # of sending an automatic response, send them the "no more today" # message. log.info('hold autoresponse limit hit: %s', sender) response_set.response_sent(address, Response.hold) # Send this notification message instead. template = getUtility(ITemplateLoader).get( 'list:user:notice:no-more-today', mlist, language=language.code) text = wrap(expand(template, mlist, dict( language=language.code, count=todays_count, sender_email=sender, # For backward compatibility. sender=sender, owneremail=mlist.owner_address, ))) with _.using(language.code): msg = UserNotification( sender, mlist.owner_address, _('Last autoresponse notification for today'), text, lang=language) msg.send(mlist) return False else: # We've sent them everything we're going to send them today. log.info('Automatic response limit discard: %s', sender) return False @public class HoldChain(TerminalChainBase): """Hold a message.""" name = 'hold' description = _('Hold a message and stop processing.') def _process(self, mlist, msg, msgdata): """See `TerminalChainBase`.""" # Start by decorating the message with a header that contains a list # of all the rules that matched. These metadata could be None or an # empty list. rule_hits = msgdata.get('rule_hits') if rule_hits: msg['X-Mailman-Rule-Hits'] = SEMISPACE.join(rule_hits) rule_misses = msgdata.get('rule_misses') if rule_misses: msg['X-Mailman-Rule-Misses'] = SEMISPACE.join(rule_misses) reasons = format_reasons(msgdata.get('moderation_reasons', ['n/a'])) # Hold the message by adding it to the list's request database. request_id = hold_message(mlist, msg, msgdata, SEMISPACE.join(reasons)) # Calculate a confirmation token to send to the author of the # message. pendable = HeldMessagePendable(id=request_id) token = getUtility(IPendings).add(pendable) # Get the language to send the response in. If the sender is a # member, then send it in the member's language, otherwise send it in # the mailing list's preferred language. member = mlist.members.get_member(msg.sender) language = (member.preferred_language if member else mlist.preferred_language) # A substitution dictionary for the email templates. charset = mlist.preferred_language.charset original_subject = msg.get('subject') if original_subject is None: original_subject = _('(no subject)') else: # This must be encoded to the mailing list's perferred charset, # ignoring incompatible characters, otherwise when creating the # notification messages, we could get a Unicode error. oneline_subject = oneline(original_subject, in_unicode=True) bytes_subject = oneline_subject.encode(charset, 'replace') original_subject = bytes_subject.decode(charset) substitutions = dict( subject=original_subject, sender_email=msg.sender, reasons=_compose_reasons(msgdata), # For backward compatibility. sender=msg.sender, ) # At this point the message is held, but now we have to craft at least # two responses. The first will go to the original author of the # message and it will contain the token allowing them to approve or # discard the message. The second one will go to the moderators of # the mailing list, if the list is so configured. # # Start by possibly sending a response to the message author. There # are several reasons why we might not go through with this. If the # message was gated from NNTP, the author may not even know about this # list, so don't spam them. If the author specifically requested that # acknowledgments not be sent, or if the message was bulk email, then # we do not send the response. It's also possible that either the # mailing list, or the author (if they are a member) have been # configured to not send such responses. if (not msgdata.get('fromusenet') and can_acknowledge(msg) and mlist.respond_to_post_requests and autorespond_to_sender(mlist, msg.sender, language)): # We can respond to the sender with a message indicating their # posting was held. subject = _( 'Your message to $mlist.fqdn_listname awaits moderator approval') send_language_code = msgdata.get('lang', language.code) template = getUtility(ITemplateLoader).get( 'list:user:notice:hold', mlist, language=send_language_code) text = wrap(expand(template, mlist, dict( language=send_language_code, **substitutions))) adminaddr = mlist.bounces_address nmsg = UserNotification( msg.sender, adminaddr, subject, text, getUtility(ILanguageManager)[send_language_code]) nmsg.send(mlist) # Now the message for the list moderators. This one should appear to # come from -owner since we really don't need to do bounce # processing on it. if mlist.admin_immed_notify: # Now let's temporarily set the language context to that which the # administrators are expecting. with _.using(mlist.preferred_language.code): language = mlist.preferred_language charset = language.charset substitutions['subject'] = original_subject # We need to regenerate or re-translate a few values in the # substitution dictionary. substitutions['reasons'] = _compose_reasons(msgdata, 55) # craft the admin notification message and deliver it subject = _( '$mlist.fqdn_listname post from $msg.sender requires ' 'approval') nmsg = UserNotification(mlist.owner_address, mlist.owner_address, subject, lang=language) nmsg.set_type('multipart/mixed') template = getUtility(ITemplateLoader).get( 'list:admin:action:post', mlist) text = MIMEText(expand(template, mlist, substitutions), _charset=charset) dmsg = MIMEText(wrap(_("""\ If you reply to this message, keeping the Subject: header intact, Mailman will discard the held message. Do this if the message is spam. If you reply to this message and include an Approved: header with the list password in it, the message will be approved for posting to the list. The Approved: header can also appear in the first line of the body of the reply.""")), _charset=language.charset) dmsg['Subject'] = 'confirm ' + token dmsg['From'] = mlist.request_address dmsg['Date'] = formatdate(localtime=True) dmsg['Message-ID'] = make_msgid() nmsg.attach(text) nmsg.attach(MIMEMessage(msg)) nmsg.attach(MIMEMessage(dmsg)) nmsg.send(mlist) # Log the held message. Log messages are not translated, so recast # the reasons in the English. with _.using('en'): reasons = format_reasons( msgdata.get('moderation_reasons', ['N/A'])) log.info('HOLD: %s post from %s held, message-id=%s: %s', mlist.fqdn_listname, msg.sender, msg.get('message-id', 'n/a'), SEMISPACE.join(reasons)) notify(HoldEvent(mlist, msg, msgdata, self)) mailman-3.2.2/src/mailman/chains/moderation.py0000644000175000017500000000547413442110351022443 0ustar maxkingmaxking00000000000000# Copyright (C) 2010-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Moderation chain. When a member or nonmember posting to the mailing list has a moderation action that is not `defer`, the built-in chain jumps to this chain. This chain then determines the disposition of the message based on the member's or nonmember's moderation action. For example, these actions jump to the appropriate terminal chain: * accept - the message is immediately accepted * hold - the message is held for moderator approval * reject - the message is bounced * discard - the message is immediately thrown away Note that if the moderation action is `defer` then the normal decisions are made as to the disposition of the message. `defer` is the default for members, while `hold` is the default for nonmembers. """ from mailman.chains.base import JumpChainBase from mailman.core.i18n import _ from mailman.interfaces.action import Action from public import public @public class ModerationChain(JumpChainBase): """Dynamically produce a link jumping to the appropriate terminal chain. The terminal chain will be one of the Accept, Hold, Discard, or Reject chains, based on the member's or nonmember's moderation action setting. """ name = 'moderation' description = _('Moderation chain') def jump_to(self, mlist, msg, msgdata): # Get the moderation action from the message metadata. It can only be # one of the expected values (i.e. not Action.defer). See the # moderation.py rule for details. This is stored in the metadata as a # string so that it can be stored in the pending table. action = Action[msgdata.get('member_moderation_action')] # defer is not a valid moderation action. jump_chain = { Action.accept: 'accept', Action.discard: 'discard', Action.hold: 'hold', Action.reject: 'reject', }.get(action) assert jump_chain is not None, ( '{}: Invalid moderation action: {} for sender: {}'.format( mlist.list_id, action, msgdata.get('moderation_sender', '(unknown)'))) return jump_chain mailman-3.2.2/src/mailman/chains/owner.py0000644000175000017500000000302613442110351021423 0ustar maxkingmaxking00000000000000# Copyright (C) 2012-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """The standard -owner posting chain.""" import logging from mailman.chains.base import TerminalChainBase from mailman.config import config from mailman.core.i18n import _ from mailman.interfaces.chain import AcceptOwnerEvent from public import public from zope.event import notify log = logging.getLogger('mailman.vette') @public class BuiltInOwnerChain(TerminalChainBase): """Default built-in -owner address chain.""" name = 'default-owner-chain' description = _('The built-in -owner posting chain.') def _process(self, mlist, msg, msgdata): # At least for now, everything posted to -owners goes through. config.switchboards['pipeline'].enqueue(msg, msgdata) log.info('OWNER: %s', msg.get('message-id', 'n/a')) notify(AcceptOwnerEvent(mlist, msg, msgdata, self)) mailman-3.2.2/src/mailman/chains/reject.py0000644000175000017500000000462013442110351021546 0ustar maxkingmaxking00000000000000# Copyright (C) 2007-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """The terminal 'reject' chain.""" import logging from mailman.app.bounces import bounce_message from mailman.chains.base import TerminalChainBase from mailman.core.i18n import _ from mailman.interfaces.chain import RejectEvent from mailman.interfaces.pipeline import RejectMessage from mailman.interfaces.template import ITemplateLoader from public import public from zope.component import getUtility from zope.event import notify log = logging.getLogger('mailman.vette') NEWLINE = '\n' SEMISPACE = '; ' @public class RejectChain(TerminalChainBase): """Reject/bounce a message.""" name = 'reject' description = _('Reject/bounce a message and stop processing.') def _process(self, mlist, msg, msgdata): """See `TerminalChainBase`.""" # Start by decorating the message with a header that contains a list # of all the rules that matched. These metadata could be None or an # empty list. rule_hits = msgdata.get('rule_hits') if rule_hits: msg['X-Mailman-Rule-Hits'] = SEMISPACE.join(rule_hits) rule_misses = msgdata.get('rule_misses') if rule_misses: msg['X-Mailman-Rule-Misses'] = SEMISPACE.join(rule_misses) reasons = msgdata.get('moderation_reasons') if reasons is None: error = None else: template = getUtility(ITemplateLoader).get( 'list:user:notice:rejected', mlist) error = RejectMessage( template, reasons, dict(listname=mlist.display_name)) bounce_message(mlist, msg, error) log.info('REJECT: %s', msg.get('message-id', 'n/a')) notify(RejectEvent(mlist, msg, msgdata, self)) mailman-3.2.2/src/mailman/chains/tests/0000755000175000017500000000000013445614541021075 5ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/chains/tests/__init__.py0000644000175000017500000000000013244427337023176 0ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/chains/tests/issue144.eml0000644000175000017500000000023613244427337023160 0ustar maxkingmaxking00000000000000To: infrastructure@lists.example.org Subject: =?UTF-8?B?VmnFoWVuYW1qZW5za2kgcGnFoXRvbGogemEgdm9kdSA4LzE=?= Message-ID: From: Ignore mailman-3.2.2/src/mailman/chains/tests/test_accept.py0000644000175000017500000000446413442110351023740 0ustar maxkingmaxking00000000000000# Copyright (C) 2016-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Test the accept chain.""" import unittest from mailman.app.lifecycle import create_list from mailman.chains.base import Link from mailman.config import config from mailman.core.chains import process as process_chain from mailman.interfaces.chain import AcceptEvent, IChain, LinkAction from mailman.testing.helpers import ( event_subscribers, specialized_message_from_string as mfs) from mailman.testing.layers import ConfigLayer from zope.interface import implementer @implementer(IChain) class MyChain: name = 'mine' description = 'A test chain' def get_links(self, mlist, msg, msgdata): def set_hits(mlist, msg, msgdata): msgdata['rule_hits'] = ['first', 'second', 'third'] yield Link('truth', LinkAction.run, function=set_hits) yield Link('truth', LinkAction.jump, 'accept') class TestAccept(unittest.TestCase): """Test the accept chain.""" layer = ConfigLayer def setUp(self): self._mlist = create_list('ant@example.com') self._msg = mfs("""\ From: anne@example.com To: ant@example.com Subject: Ignore """) def test_rule_hits(self): config.chains['mine'] = MyChain() self.addCleanup(config.chains.pop, 'mine') hits = None def handler(event): # noqa: E306 nonlocal hits if isinstance(event, AcceptEvent): hits = event.msg['x-mailman-rule-hits'] with event_subscribers(handler): process_chain(self._mlist, self._msg, {}, start_chain='mine') self.assertEqual(hits, 'first; second; third') mailman-3.2.2/src/mailman/chains/tests/test_base.py0000644000175000017500000000571413442110351023412 0ustar maxkingmaxking00000000000000# Copyright (C) 2014-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Test the base chain stuff.""" import unittest from mailman.chains.accept import AcceptChain from mailman.chains.base import Chain, Link, TerminalChainBase from mailman.interfaces.chain import LinkAction from mailman.rules.any import Any from mailman.testing.layers import ConfigLayer class SimpleChain(TerminalChainBase): def _process(self, mlist, msg, msgdata): pass class TestMiscellaneous(unittest.TestCase): """Reach additional code coverage.""" def test_link_repr(self): self.assertEqual( repr(Link(Any())), '') def test_link_repr_function(self): def function(): pass self.assertEqual( repr(Link(Any(), function=function)), '') def test_link_repr_chain(self): self.assertEqual( repr(Link(Any(), chain=AcceptChain())), '') def test_link_repr_chain_and_function(self): def function(): pass self.assertEqual( repr(Link(Any(), chain=AcceptChain(), function=function)), '') def test_link_repr_chain_all(self): def function(): pass self.assertEqual( repr(Link(Any(), LinkAction.stop, AcceptChain(), function)), '') def test_flush(self): # Test that we can flush the links of a chain. chain = Chain('test', 'just a testing chain') chain.append_link(Link(Any())) # Iterate over the links of the chain to prove there are some. count = sum(1 for link in chain.get_iterator()) self.assertEqual(count, 1) # Flush the chain; then there will be no links. chain.flush() count = sum(1 for link in chain.get_iterator()) self.assertEqual(count, 0) class TestTerminalChainBase(unittest.TestCase): layer = ConfigLayer def test_terminal_chain_iterator(self): chain = SimpleChain() self.assertEqual([link.action for link in chain], [LinkAction.run, LinkAction.stop]) mailman-3.2.2/src/mailman/chains/tests/test_dmarc.py0000644000175000017500000000563413442110351023567 0ustar maxkingmaxking00000000000000# Copyright (C) 2017-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Test for the DMARC chain.""" import unittest from mailman.app.lifecycle import create_list from mailman.core.chains import process as process_chain from mailman.interfaces.chain import DiscardEvent, RejectEvent from mailman.testing.helpers import ( event_subscribers, get_queue_messages, specialized_message_from_string as mfs) from mailman.testing.layers import ConfigLayer class TestDMARC(unittest.TestCase): layer = ConfigLayer maxDiff = None def setUp(self): self._mlist = create_list('ant@example.com') self._msg = mfs("""\ From: anne@example.com To: test@example.com Subject: Ignore """) def test_discard(self): msgdata = dict(dmarc_action='discard') # When a message is discarded, the only artifacts are a log message # and an event. Catch the event to prove it happened. events = [] def handler(event): # noqa: E306 if isinstance(event, DiscardEvent): events.append(event) with event_subscribers(handler): process_chain(self._mlist, self._msg, msgdata, start_chain='dmarc') self.assertEqual(len(events), 1) self.assertIs(events[0].msg, self._msg) def test_reject(self): msgdata = dict( dmarc_action='reject', moderation_reasons=['DMARC violation'], ) # When a message is reject, an event will be triggered and the message # will be bounced. events = [] def handler(event): # noqa: E306 if isinstance(event, RejectEvent): events.append(event) with event_subscribers(handler): process_chain(self._mlist, self._msg, msgdata, start_chain='dmarc') self.assertEqual(len(events), 1) self.assertIs(events[0].msg, self._msg) items = get_queue_messages('virgin', expected_count=1) # Unpack the rejection message. rejection = items[0].msg.get_payload(0).get_payload() self.assertEqual(rejection, """\ Your message to the Ant mailing-list was rejected for the following reasons: DMARC violation The original message as received by Mailman is attached. """) mailman-3.2.2/src/mailman/chains/tests/test_headers.py0000644000175000017500000004050013442110351024103 0ustar maxkingmaxking00000000000000# Copyright (C) 2012-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Test the header chain.""" import unittest from email import message_from_bytes from mailman.app.lifecycle import create_list from mailman.chains.headers import HeaderMatchRule, make_link from mailman.config import config from mailman.core.chains import process from mailman.email.message import Message from mailman.interfaces.chain import ( DiscardEvent, HoldEvent, LinkAction, RejectEvent) from mailman.interfaces.mailinglist import IHeaderMatchList from mailman.testing.helpers import ( LogFileMark, configuration, event_subscribers, specialized_message_from_string as mfs) from mailman.testing.layers import ConfigLayer class TestHeaderChain(unittest.TestCase): """Test the header chain code.""" layer = ConfigLayer def setUp(self): self._mlist = create_list('test@example.com') def test_make_link(self): # Test that make_link() with no given chain creates a Link with a # deferred link action. link = make_link('Subject', '[tT]esting') self.assertEqual(link.rule.header, 'Subject') self.assertEqual(link.rule.pattern, '[tT]esting') self.assertEqual(link.action, LinkAction.defer) self.assertIsNone(link.chain) def test_make_link_with_chain(self): # Test that make_link() with a given chain creates a Link with a jump # action to the chain. link = make_link('Subject', '[tT]esting', 'accept') self.assertEqual(link.rule.header, 'Subject') self.assertEqual(link.rule.pattern, '[tT]esting') self.assertEqual(link.action, LinkAction.jump) self.assertEqual(link.chain, config.chains['accept']) @configuration('antispam', header_checks=""" Foo: a+ Bar: bb? """) def test_config_checks(self): # Test that the header-match chain has the header checks from the # configuration file. chain = config.chains['header-match'] # The links are created dynamically; the rule names will all start # with the same prefix, but have a variable suffix. The actions will # all be to jump to the named chain. Do these checks now, while we # collect other useful information. post_checks = [] saw_any_rule = False for link in chain.get_links(self._mlist, Message(), {}): if link.rule.name == 'any': saw_any_rule = True self.assertEqual(link.action, LinkAction.jump) elif saw_any_rule: raise AssertionError("'any' rule was not last") else: self.assertEqual(link.rule.name[:13], 'header-match-') self.assertEqual(link.action, LinkAction.defer) post_checks.append((link.rule.header, link.rule.pattern)) self.assertListEqual(post_checks, [ ('Foo', 'a+'), ('Bar', 'bb?'), ]) @configuration('antispam', header_checks=""" Foo: foo A-bad-line Bar: bar """) def test_bad_configuration_line(self): # Take a mark on the error log file. mark = LogFileMark('mailman.error') # A bad value in [antispam]header_checks should just get ignored, but # with an error message logged. chain = config.chains['header-match'] # The links are created dynamically; the rule names will all start # with the same prefix, but have a variable suffix. The actions will # all be to jump to the named chain. Do these checks now, while we # collect other useful information. post_checks = [] saw_any_rule = False for link in chain.get_links(self._mlist, Message(), {}): if link.rule.name == 'any': saw_any_rule = True self.assertEqual(link.action, LinkAction.jump) elif saw_any_rule: raise AssertionError("'any' rule was not last") else: self.assertEqual(link.rule.name[:13], 'header-match-') self.assertEqual(link.action, LinkAction.defer) post_checks.append((link.rule.header, link.rule.pattern)) self.assertListEqual(post_checks, [ ('Foo', 'foo'), ('Bar', 'bar'), ]) # Check the error log. self.assertEqual(mark.readline()[-77:-1], 'Configuration error: [antispam]header_checks ' 'contains bogus line: A-bad-line') def test_bad_regexp(self): # Take a mark on the error log file. mark = LogFileMark('mailman.error') # A bad regexp in header_checks should just get ignored, but # with an error message logged. header_matches = IHeaderMatchList(self._mlist) header_matches.append('Foo', '+a bad regexp', 'reject') msg = mfs("""\ From: anne@example.com To: test@example.com Subject: A message Message-ID: Foo: foo MIME-Version: 1.0 A message body. """) msgdata = {} # This event subscriber records the event that occurs when the message # is processed by the header-match chain. events = [] with event_subscribers(events.append): process(self._mlist, msg, msgdata, start_chain='header-match') self.assertEqual(len(events), 0) # Check the error log. self.assertEqual(mark.readline()[-89:-1], "Invalid regexp '+a bad regexp' in header_matches " 'for test.example.com: nothing to repeat') def test_duplicate_header_match_rule(self): # 100% coverage: test an assertion in a corner case. # # Save the existing rules so they can be restored later. saved_rules = config.rules.copy() self.addCleanup(setattr, config, 'rules', saved_rules) HeaderMatchRule('x-spam-score', '*', suffix='100') self.assertRaises(AssertionError, HeaderMatchRule, 'x-spam-score', '.*', suffix='100') def test_list_rule(self): # Test that the header-match chain has the header checks from the # mailing-list configuration. chain = config.chains['header-match'] header_matches = IHeaderMatchList(self._mlist) header_matches.append('Foo', 'a+') links = [link for link in chain.get_links(self._mlist, Message(), {}) if link.rule.name != 'any'] self.assertEqual(len(links), 1) self.assertEqual(links[0].action, LinkAction.jump) self.assertEqual(links[0].chain.name, config.antispam.jump_chain) self.assertEqual(links[0].rule.header, 'foo') self.assertEqual(links[0].rule.pattern, 'a+') self.assertTrue(links[0].rule.name.startswith( 'header-match-test.example.com-')) def test_list_complex_rule(self): # Test that the mailing-list header-match complex rules are read # properly. chain = config.chains['header-match'] header_matches = IHeaderMatchList(self._mlist) header_matches.append('Foo', 'a+', 'reject') header_matches.append('Bar', 'b+', 'discard') header_matches.append('Baz', 'z+', 'accept') links = [link for link in chain.get_links(self._mlist, Message(), {}) if link.rule.name != 'any'] self.assertEqual(len(links), 3) self.assertEqual([ (link.rule.header, link.rule.pattern, link.action, link.chain.name) for link in links ], [('foo', 'a+', LinkAction.jump, 'reject'), ('bar', 'b+', LinkAction.jump, 'discard'), ('baz', 'z+', LinkAction.jump, 'accept'), ]) # noqa: E124 def test_header_in_subpart(self): # Test that headers in sub-parts are also matched. msg = mfs("""\ From: anne@example.com To: test@example.com Subject: A message Message-ID: Foo: foo MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="================12345==" --================12345== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit A message body. --================12345== Content-Type: application/junk MIME-Version: 1.0 Content-Transfer-Encoding: 7bit This is junk --================12345==-- """) msgdata = {} header_matches = IHeaderMatchList(self._mlist) header_matches.append('Content-Type', 'application/junk', 'hold') # This event subscriber records the event that occurs when the message # is processed by the owner chain. events = [] with event_subscribers(events.append): process(self._mlist, msg, msgdata, start_chain='header-match') self.assertEqual(len(events), 1) event = events[0] self.assertIsInstance(event, HoldEvent) self.assertEqual(event.chain, config.chains['hold']) def test_get_all_returns_non_string(self): # Test case where msg.get_all() returns header instance. msg = message_from_bytes(b"""\ From: anne@example.com To: test@example.com Subject: Bad \x96 subject Message-ID: body """, Message) msgdata = {} header_matches = IHeaderMatchList(self._mlist) header_matches.append('Subject', 'Bad', 'hold') # This event subscriber records the event that occurs when the message # is processed by the owner chain. events = [] with event_subscribers(events.append): process(self._mlist, msg, msgdata, start_chain='header-match') self.assertEqual(len(events), 1) event = events[0] self.assertIsInstance(event, HoldEvent) self.assertEqual(event.chain, config.chains['hold']) @configuration('antispam', header_checks=""" Foo: foo """, jump_chain='hold') def test_priority_site_over_list(self): # Test that the site-wide checks take precedence over the list-specific # checks. msg = mfs("""\ From: anne@example.com To: test@example.com Subject: A message Message-ID: Foo: foo MIME-Version: 1.0 A message body. """) msgdata = {} header_matches = IHeaderMatchList(self._mlist) header_matches.append('Foo', 'foo', 'accept') # This event subscriber records the event that occurs when the message # is processed by the owner chain. events = [] with event_subscribers(events.append): process(self._mlist, msg, msgdata, start_chain='header-match') self.assertEqual(len(events), 1) event = events[0] # Site-wide wants to hold the message, the list wants to accept it. self.assertIsInstance(event, HoldEvent) self.assertEqual(event.chain, config.chains['hold']) def test_no_action_defaults_to_site_wide_action(self): # If the list-specific header check matches, but there is no defined # action, the site-wide antispam action is used. msg = mfs("""\ From: anne@example.com To: test@example.com Subject: A message Message-ID: Foo: foo MIME-Version: 1.0 A message body. """) header_matches = IHeaderMatchList(self._mlist) header_matches.append('Foo', 'foo') # This event subscriber records the event that occurs when the message # is processed by the owner chain, which holds its for approval. events = [] def record_holds(event): # noqa: E301 if not isinstance(event, HoldEvent): return events.append(event) with event_subscribers(record_holds): # Set the site-wide antispam action to hold the message. with configuration('antispam', header_checks=""" Spam: [*]{3,} """, jump_chain='hold'): # noqa: E125 process(self._mlist, msg, {}, start_chain='header-match') self.assertEqual(len(events), 1) event = events[0] self.assertIsInstance(event, HoldEvent) self.assertEqual(event.chain, config.chains['hold']) self.assertEqual(event.mlist, self._mlist) self.assertEqual(event.msg, msg) events = [] def record_discards(event): # noqa: E301 if not isinstance(event, DiscardEvent): return events.append(event) with event_subscribers(record_discards): # Set the site-wide default to discard the message. msg.replace_header('Message-Id', '') with configuration('antispam', header_checks=""" Spam: [*]{3,} """, jump_chain='discard'): # noqa: E125 process(self._mlist, msg, {}, start_chain='header-match') self.assertEqual(len(events), 1) event = events[0] self.assertIsInstance(event, DiscardEvent) self.assertEqual(event.chain, config.chains['discard']) self.assertEqual(event.mlist, self._mlist) self.assertEqual(event.msg, msg) @configuration('antispam', header_checks=""" Header1: a+ """, jump_chain='hold') def test_reuse_rules(self): # Test that existing header-match rules are used instead of creating # new ones. chain = config.chains['header-match'] header_matches = IHeaderMatchList(self._mlist) header_matches.append('Header2', 'b+') header_matches.append('Header3', 'c+') def get_links(): # noqa: E306 return [ link for link in chain.get_links(self._mlist, Message(), {}) if link.rule.name != 'any' ] links_1 = get_links() self.assertEqual(len(links_1), 3) links_2 = get_links() # The link rules both have the same name... self.assertEqual( [l.rule.name for l in links_1], [l.rule.name for l in links_2], ) # ...and are actually the identical objects. for link1, link2 in zip(links_1, links_2): self.assertIs(link1.rule, link2.rule) def test_hold_returns_reason(self): # Test that a match with hold action returns a reason msg = mfs("""\ From: anne@example.com To: test@example.com Subject: Bad subject Message-ID: body """) msgdata = {} header_matches = IHeaderMatchList(self._mlist) header_matches.append('Subject', 'Bad', 'hold') # This event subscriber records the event that occurs when the message # is processed by the owner chain. events = [] with event_subscribers(events.append): process(self._mlist, msg, msgdata, start_chain='header-match') self.assertEqual(len(events), 1) event = events[0] self.assertIsInstance(event, HoldEvent) self.assertEqual(msgdata['moderation_reasons'], [('Header "{}" matched a header rule', 'Bad subject')]) def test_reject_returns_reason(self): # Test that a match with reject action returns a reason msg = mfs("""\ From: anne@example.com To: test@example.com Subject: Bad subject Message-ID: body """) msgdata = {} header_matches = IHeaderMatchList(self._mlist) header_matches.append('Subject', 'Bad', 'reject') # This event subscriber records the event that occurs when the message # is processed by the owner chain. events = [] with event_subscribers(events.append): process(self._mlist, msg, msgdata, start_chain='header-match') self.assertEqual(len(events), 1) event = events[0] self.assertIsInstance(event, RejectEvent) self.assertEqual(msgdata['moderation_reasons'], [('Header "{}" matched a header rule', 'Bad subject')]) mailman-3.2.2/src/mailman/chains/tests/test_hold.py0000644000175000017500000002463113442110351023425 0ustar maxkingmaxking00000000000000# Copyright (C) 2011-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Additional tests for the hold chain.""" import unittest from email import message_from_bytes as mfb from importlib_resources import read_binary from mailman.app.lifecycle import create_list from mailman.chains.builtin import BuiltInChain from mailman.chains.hold import HoldChain, autorespond_to_sender from mailman.core.chains import process as process_chain from mailman.interfaces.autorespond import IAutoResponseSet, Response from mailman.interfaces.member import MemberRole from mailman.interfaces.messages import IMessageStore from mailman.interfaces.requests import IListRequests, RequestType from mailman.interfaces.usermanager import IUserManager from mailman.testing.helpers import ( LogFileMark, configuration, get_queue_messages, set_preferred, specialized_message_from_string as mfs) from mailman.testing.layers import ConfigLayer from zope.component import getUtility SEMISPACE = '; ' class TestAutorespond(unittest.TestCase): """Test autorespond_to_sender()""" layer = ConfigLayer maxDiff = None def setUp(self): self._mlist = create_list('test@example.com') @configuration('mta', max_autoresponses_per_day=1) def test_max_autoresponses_per_day(self): # The last one we sent was the last one we should send today. Instead # of sending an automatic response, send them the "no more today" # message. Start by simulating a response having been sent to an # address already. anne = getUtility(IUserManager).create_address('anne@example.com') response_set = IAutoResponseSet(self._mlist) response_set.response_sent(anne, Response.hold) # Trigger the sending of a "last response for today" using the default # language (i.e. the mailing list's preferred language). autorespond_to_sender(self._mlist, 'anne@example.com') # So first, there should be one more hold response sent to the user. self.assertEqual(response_set.todays_count(anne, Response.hold), 2) # And the virgin queue should have the message in it. messages = get_queue_messages('virgin') self.assertEqual(len(messages), 1) # Remove the variable headers. message = messages[0].msg self.assertIn('message-id', message) del message['message-id'] self.assertIn('date', message) del message['date'] self.assertMultiLineEqual(messages[0].msg.as_string(), """\ MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Subject: Last autoresponse notification for today From: test-owner@example.com To: anne@example.com Precedence: bulk We have received a message from your address requesting an automated response from the test@example.com mailing list. The number we have seen today: 1. In order to avoid problems such as mail loops between email robots, we will not be sending you any further responses today. Please try again tomorrow. If you believe this message is in error, or if you have any questions, please contact the list owner at test-owner@example.com.""") class TestHoldChain(unittest.TestCase): """Test the hold chain code.""" layer = ConfigLayer def setUp(self): self._mlist = create_list('test@example.com') self._user_manager = getUtility(IUserManager) def test_hold_chain(self): msg = mfs("""\ From: anne@example.com To: test@example.com Subject: A message Message-ID: MIME-Version: 1.0 A message body. """) msgdata = dict(moderation_reasons=[ 'TEST-REASON-1', 'TEST-REASON-2', ('TEST-{}-REASON-{}', 'FORMAT', 3), ]) logfile = LogFileMark('mailman.vette') process_chain(self._mlist, msg, msgdata, start_chain='hold') messages = get_queue_messages('virgin', expected_count=2) payloads = {} for item in messages: if item.msg['to'] == 'test-owner@example.com': part = item.msg.get_payload(0) payloads['owner'] = part.get_payload().splitlines() elif item.msg['To'] == 'anne@example.com': payloads['sender'] = item.msg.get_payload().splitlines() else: self.fail('Unexpected message: %s' % item.msg) self.assertIn(' TEST-REASON-1', payloads['owner']) self.assertIn(' TEST-REASON-2', payloads['owner']) self.assertIn(' TEST-FORMAT-REASON-3', payloads['owner']) self.assertIn(' TEST-REASON-1', payloads['sender']) self.assertIn(' TEST-REASON-2', payloads['sender']) self.assertIn(' TEST-FORMAT-REASON-3', payloads['sender']) logged = logfile.read() self.assertIn('TEST-REASON-1', logged) self.assertIn('TEST-REASON-2', logged) self.assertIn('TEST-FORMAT-REASON-3', logged) # Check the reason passed to hold_message(). requests = IListRequests(self._mlist) self.assertEqual(requests.count_of(RequestType.held_message), 1) request = requests.of_type(RequestType.held_message)[0] key, data = requests.get_request(request.id) self.assertEqual( data.get('_mod_reason'), 'TEST-REASON-1; TEST-REASON-2; TEST-FORMAT-REASON-3') def test_hold_chain_no_reasons_given(self): msg = mfs("""\ From: anne@example.com To: test@example.com Subject: A message Message-ID: MIME-Version: 1.0 A message body. """) process_chain(self._mlist, msg, {}, start_chain='hold') # No reason was given, so a default is used. requests = IListRequests(self._mlist) self.assertEqual(requests.count_of(RequestType.held_message), 1) request = requests.of_type(RequestType.held_message)[0] key, data = requests.get_request(request.id) self.assertEqual(data.get('_mod_reason'), 'n/a') def test_hold_chain_charset(self): # Issue #144 - UnicodeEncodeError in the hold chain. self._mlist.admin_immed_notify = True self._mlist.respond_to_post_requests = False bart = self._user_manager.create_user('bart@example.com', 'Bart User') address = set_preferred(bart) self._mlist.subscribe(address, MemberRole.moderator) msg = mfb(read_binary('mailman.chains.tests', 'issue144.eml')) msg.sender = 'anne@example.com' process_chain(self._mlist, msg, {}, start_chain='hold') # The postauth.txt message is now in the virgin queue awaiting # delivery to the moderators. items = get_queue_messages('virgin', expected_count=1) msgdata = items[0].msgdata # Should get sent to -owner address. self.assertEqual(msgdata['recipients'], {'test-owner@example.com'}) # Ensure that the subject looks correct in the postauth.txt. msg = items[0].msg value = None for line in msg.get_payload(0).get_payload().splitlines(): if line.strip().startswith('Subject:'): header, colon, value = line.partition(':') break self.assertEqual(value.lstrip(), 'Vi?enamjenski pi?tolj za vodu 8/1') self.assertEqual( msg['Subject'], 'test@example.com post from anne@example.com requires approval') def test_hold_chain_crosspost(self): mlist2 = create_list('test2@example.com') msg = mfs("""\ From: anne@example.com To: test@example.com, test2@example.com Subject: A message Message-ID: MIME-Version: 1.0 A message body. """) process_chain(self._mlist, msg, {}, start_chain='hold') process_chain(mlist2, msg, {}, start_chain='hold') # There are four items in the virgin queue. Two of them are for the # list owners who need to moderate the held message, and the other is # for anne telling her that her message was held for approval. items = get_queue_messages('virgin', expected_count=4) anne_froms = set() owner_tos = set() for item in items: if item.msg['to'] == 'anne@example.com': anne_froms.add(item.msg['from']) else: owner_tos.add(item.msg['to']) self.assertEqual(anne_froms, set(['test-bounces@example.com', 'test2-bounces@example.com'])) self.assertEqual(owner_tos, set(['test-owner@example.com', 'test2-owner@example.com'])) # And the message appears in the store. messages = list(getUtility(IMessageStore).messages) self.assertEqual(len(messages), 1) self.assertEqual(messages[0]['message-id'], '') def test_hold_with_long_rule_misses(self): msg = mfs("""\ From: anne@example.com To: test@example.com Subject: A message Message-ID: MIME-Version: 1.0 A message body. """) rule_misses = [x[0] for x in BuiltInChain._link_descriptions if x[0] not in ('truth', 'any')] for i in range(20): rule_misses.append('header-match-test.example.com-{}'.format(i)) msgdata = dict(rule_misses=rule_misses) msgdata['rule_hits'] = [] msgdata['moderation_reasons'] = ['something'] # We can't use process_chain because it clears rule hits and misses. HoldChain()._process(self._mlist, msg, msgdata) messages = get_queue_messages('virgin', expected_count=2) for item in messages: if item.msg['to'] == 'test-owner@example.com': held_message = item.msg.get_payload(1).get_payload(0) elif item.msg['To'] == 'anne@example.com': pass else: self.fail('Unexpected message: %s' % item.msg) self.assertEqual(held_message['x-mailman-rule-misses'], SEMISPACE.join(rule_misses)) mailman-3.2.2/src/mailman/chains/tests/test_owner.py0000644000175000017500000000463713442110351023635 0ustar maxkingmaxking00000000000000# Copyright (C) 2012-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Test the owner chain.""" import unittest from mailman.app.lifecycle import create_list from mailman.chains.owner import BuiltInOwnerChain from mailman.core.chains import process from mailman.interfaces.chain import AcceptOwnerEvent from mailman.testing.helpers import ( event_subscribers, get_queue_messages, specialized_message_from_string as mfs) from mailman.testing.layers import ConfigLayer class TestOwnerChain(unittest.TestCase): """Test the owner chain.""" layer = ConfigLayer def setUp(self): self._mlist = create_list('test@example.com') self._msg = mfs("""\ From: anne@example.com To: test@example.com Message-ID: """) def test_owner_pipeline(self): # Messages processed through the default owners chain end up in the # pipeline queue, and an event gets sent. # # This event subscriber records the event that occurs when the message # is processed by the owner chain. events = [] def catch_event(event): # noqa: E306 if isinstance(event, AcceptOwnerEvent): events.append(event) with event_subscribers(catch_event): process(self._mlist, self._msg, {}, 'default-owner-chain') self.assertEqual(len(events), 1) event = events[0] self.assertIsInstance(event, AcceptOwnerEvent) self.assertEqual(event.mlist, self._mlist) self.assertEqual(event.msg['message-id'], '') self.assertIsInstance(event.chain, BuiltInOwnerChain) items = get_queue_messages('pipeline', expected_count=1) message = items[0].msg self.assertEqual(message['message-id'], '') mailman-3.2.2/src/mailman/chains/tests/test_reject.py0000644000175000017500000000433013442110351023745 0ustar maxkingmaxking00000000000000# Copyright (C) 2015-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Test the reject chain.""" import unittest from mailman.app.lifecycle import create_list from mailman.core.chains import process as process_chain from mailman.testing.helpers import ( get_queue_messages, specialized_message_from_string as mfs) from mailman.testing.layers import ConfigLayer class TestReject(unittest.TestCase): """Test the reject chain.""" layer = ConfigLayer def setUp(self): self._mlist = create_list('test@example.com') self._msg = mfs("""\ From: anne@example.com To: test@example.com Subject: Ignore """) def test_reject_reasons(self): # The bounce message must contain the moderation reasons. msgdata = dict(moderation_reasons=[ 'TEST-REASON-1', 'TEST-REASON-2', ('TEST-{}-REASON-{}', 'FORMAT', 3), ]) process_chain(self._mlist, self._msg, msgdata, start_chain='reject') bounces = get_queue_messages('virgin', expected_count=1) payload = bounces[0].msg.get_payload(0).as_string() self.assertIn('TEST-REASON-1', payload) self.assertIn('TEST-REASON-2', payload) self.assertIn('TEST-FORMAT-REASON-3', payload) def test_no_reason(self): # There may be no moderation reasons. process_chain(self._mlist, self._msg, {}, start_chain='reject') bounces = get_queue_messages('virgin', expected_count=1) payload = bounces[0].msg.get_payload(0).as_string() self.assertIn('No bounce details are available', payload) mailman-3.2.2/src/mailman/commands/0000755000175000017500000000000013445614541020267 5ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/commands/__init__.py0000644000175000017500000000000013244427337022370 0ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/commands/cli_aliases.py0000644000175000017500000000306213442110351023075 0ustar maxkingmaxking00000000000000# Copyright (C) 2009-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Generate Mailman alias files for your MTA.""" import click from mailman.config import config from mailman.core.i18n import _ from mailman.interfaces.command import ICLISubCommand from mailman.utilities.modules import call_name from mailman.utilities.options import I18nCommand from public import public from zope.interface import implementer @click.command( cls=I18nCommand, help=_('Regenerate the aliases appropriate for your MTA.')) @click.option( '--directory', '-d', type=click.Path(exists=True, file_okay=False, resolve_path=True, writable=True), help=_('An alternative directory to output the various MTA files to.')) def aliases(directory): call_name(config.mta.incoming).regenerate(directory) @public @implementer(ICLISubCommand) class Aliases: name = 'aliases' command = aliases mailman-3.2.2/src/mailman/commands/cli_conf.py0000644000175000017500000001040213442110351022375 0ustar maxkingmaxking00000000000000# Copyright (C) 2013-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Print the mailman configuration.""" import click from lazr.config._config import Section from mailman.config import config from mailman.core.i18n import _ from mailman.interfaces.command import ICLISubCommand from mailman.utilities.options import I18nCommand from public import public from zope.interface import implementer def _section_exists(section): # Not all of the attributes in config are actual sections, so we have to # check the section's type. return ( hasattr(config, section) and isinstance(getattr(config, section), Section) ) def _get_value(section, key): return getattr(getattr(config, section), key) def _print_values_for_section(section, output): current_section = sorted(getattr(config, section)) for key in current_section: value = _get_value(section, key) print('[{}] {}: {}'.format(section, key, value), file=output) @click.command( cls=I18nCommand, help=_('Print the Mailman configuration.')) @click.option( '--output', '-o', type=click.File(mode='w', encoding='utf-8', atomic=True), help=_("""\ File to send the output to. If not given, or if '-' is given, standard output is used.""")) @click.option( '--section', '-s', help=_("""\ Section to use for the lookup. If no key is given, all the key-value pairs of the given section will be displayed.""")) @click.option( '--key', '-k', help=_("""\ Key to use for the lookup. If no section is given, all the key-values pair from any section matching the given key will be displayed.""")) @click.pass_context def conf(ctx, output, section, key): # Case 1: Both section and key are given, so we can look the value up # directly. if section is not None and key is not None: if not _section_exists(section): ctx.fail('No such section: {}'.format(section)) elif not hasattr(getattr(config, section), key): ctx.fail('Section {}: No such key: {}'.format(section, key)) else: print(_get_value(section, key), file=output) # Case 2: Section is given, key is not given. elif section is not None and key is None: if _section_exists(section): _print_values_for_section(section, output) else: ctx.fail('No such section: {}'.format(section)) # Case 3: Section is not given, key is given. elif section is None and key is not None: for current_section in sorted(config.schema._section_schemas): # We have to ensure that the current section actually exists # and that it contains the given key. if (_section_exists(current_section) and hasattr(getattr(config, current_section), key)): value = _get_value(current_section, key) print('[{}] {}: {}'.format( current_section, key, value), file=output) # Case 4: Neither section nor key are given, just display all the # sections and their corresponding key/value pairs. elif section is None and key is None: for current_section in sorted(config.schema._section_schemas): # However, we have to make sure that the current sections and # key which are being looked up actually exist before trying # to print them. if _section_exists(current_section): _print_values_for_section(current_section, output) else: raise AssertionError('Unexpected combination') @public @implementer(ICLISubCommand) class Conf: name = 'conf' command = conf mailman-3.2.2/src/mailman/commands/cli_control.py0000644000175000017500000001573313442110351023144 0ustar maxkingmaxking00000000000000# Copyright (C) 2009-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Start/stop/reopen/restart commands.""" import os import sys import click import errno import signal import logging from mailman.bin.master import WatcherState, master_state from mailman.config import config from mailman.core.i18n import _ from mailman.interfaces.command import ICLISubCommand from mailman.utilities.options import I18nCommand from public import public from zope.interface import implementer qlog = logging.getLogger('mailman.runner') @click.command( cls=I18nCommand, help=_('Start the Mailman master and runner processes.')) @click.option( '--force', '-f', is_flag=True, default=False, help=_("""\ If the master watcher finds an existing master lock, it will normally exit with an error message. With this option, the master will perform an extra level of checking. If a process matching the host/pid described in the lock file is running, the master will still exit, requiring you to manually clean up the lock. But if no matching process is found, the master will remove the apparently stale lock and make another attempt to claim the master lock.""")) @click.option( '--run-as-user', '-u', is_flag=True, default=True, help=_("""\ Normally, this script will refuse to run if the user id and group id are not set to the 'mailman' user and group (as defined when you configured Mailman). If run as root, this script will change to this user and group before the check is made. This can be inconvenient for testing and debugging purposes, so the -u flag means that the step that sets and checks the uid/gid is skipped, and the program is run as the current user and group. This flag is not recommended for normal production environments. Note though, that if you run with -u and are not in the mailman group, you may have permission problems, such as being unable to delete a list's archives through the web. Tough luck!""")) @click.option( '--quiet', '-q', is_flag=True, default=False, help=_("""\ Don't print status messages. Error messages are still printed to standard error.""")) @click.pass_context def start(ctx, force, run_as_user, quiet): # Although there's a potential race condition here, it's a better user # experience for the parent process to refuse to start twice, rather than # having it try to start the master, which will error exit. status, lock = master_state() if status is WatcherState.conflict: ctx.fail(_('GNU Mailman is already running')) elif status in (WatcherState.stale_lock, WatcherState.host_mismatch): if not force: ctx.fail( _('A previous run of GNU Mailman did not exit ' 'cleanly ({}). Try using --force'.format(status.name))) # Daemon process startup according to Stevens, Advanced Programming in the # UNIX Environment, Chapter 13. pid = os.fork() if pid: # parent if not quiet: print(_("Starting Mailman's master runner")) return # child: Create a new session and become the session leader, but since we # won't be opening any terminal devices, don't do the ultra-paranoid # suggestion of doing a second fork after the setsid() call. os.setsid() # Instead of cd'ing to root, cd to the Mailman runtime directory. However, # before we do that, set an environment variable used by the subprocesses # to calculate their path to the $VAR_DIR. os.environ['MAILMAN_VAR_DIR'] = config.VAR_DIR os.chdir(config.VAR_DIR) # Exec the master watcher. execl_args = [ sys.executable, sys.executable, os.path.join(config.BIN_DIR, 'master'), ] if force: execl_args.append('--force') # Always pass the configuration file path to the master process, so there's # no confusion about which one is being used. execl_args.extend(['-C', config.filename]) qlog.debug('starting: %s', execl_args) os.execl(*execl_args) # We should never get here. raise RuntimeError('os.execl() failed') @public @implementer(ICLISubCommand) class Start: name = 'start' command = start def kill_watcher(sig): try: with open(config.PID_FILE) as fp: pid = int(fp.read().strip()) except (IOError, ValueError) as error: # For i18n convenience print(_('PID unreadable in: $config.PID_FILE'), file=sys.stderr) print(error, file=sys.stderr) print(_('Is the master even running?'), file=sys.stderr) return try: os.kill(pid, sig) except OSError as error: if error.errno != errno.ESRCH: raise print(_('No child with pid: $pid'), file=sys.stderr) print(error, file=sys.stderr) print(_('Stale pid file removed.'), file=sys.stderr) os.unlink(config.PID_FILE) @click.command( cls=I18nCommand, help=_('Stop the Mailman master and runner processes.')) @click.option( '--quiet', '-q', is_flag=True, default=False, help=_("""\ Don't print status messages. Error messages are still printed to standard error.""")) def stop(quiet): if not quiet: print(_("Shutting down Mailman's master runner")) kill_watcher(signal.SIGTERM) @public @implementer(ICLISubCommand) class Stop: name = 'stop' command = stop @click.command( cls=I18nCommand, help=_('Signal the Mailman processes to re-open their log files.')) @click.option( '--quiet', '-q', is_flag=True, default=False, help=_("""\ Don't print status messages. Error messages are still printed to standard error.""")) def reopen(quiet): if not quiet: print(_('Reopening the Mailman runners')) kill_watcher(signal.SIGHUP) @public @implementer(ICLISubCommand) class Reopen: name = 'reopen' command = reopen @click.command( cls=I18nCommand, help=_('Stop and restart the Mailman runner subprocesses.')) @click.option( '--quiet', '-q', is_flag=True, default=False, help=_("""\ Don't print status messages. Error messages are still printed to standard error.""")) def restart(quiet): if not quiet: print(_('Restarting the Mailman runners')) kill_watcher(signal.SIGUSR1) @public @implementer(ICLISubCommand) class Restart: name = 'restart' command = restart mailman-3.2.2/src/mailman/commands/cli_digests.py0000644000175000017500000001055413442110351023122 0ustar maxkingmaxking00000000000000# Copyright (C) 2015-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """The `send_digests` subcommand.""" import sys import click from mailman.app.digests import ( bump_digest_number_and_volume, maybe_send_digest_now) from mailman.core.i18n import _ from mailman.interfaces.command import ICLISubCommand from mailman.interfaces.listmanager import IListManager from mailman.utilities.options import I18nCommand from public import public from zope.component import getUtility from zope.interface import implementer @click.command( cls=I18nCommand, help=_('Operate on digests.')) @click.option( '--list', '-l', 'list_ids', metavar='list', multiple=True, help=_("""\ Operate on this mailing list. Multiple --list options can be given. The argument can either be a List-ID or a fully qualified list name. Without this option, operate on the digests for all mailing lists.""")) @click.option( '--send', '-s', is_flag=True, default=False, help=_("""\ Send any collected digests right now, even if the size threshold has not yet been met.""")) @click.option( '--bump', '-b', is_flag=True, default=False, help=_("""\ Increment the digest volume number and reset the digest number to one. If given with --send, the volume number is incremented before any current digests are sent.""")) @click.option( '--dry-run', '-n', is_flag=True, default=False, help=_("""\ Don't actually do anything, but in conjunction with --verbose, show what would happen.""")) @click.option( '--verbose', '-v', is_flag=True, default=False, help=_('Print some additional status.')) @click.option( '--periodic', '-p', is_flag=True, default=False, help=_("""\ Send any collected digests for the List only if their digest_send_periodic is set to True.""")) @click.pass_context def digests(ctx, list_ids, send, bump, dry_run, verbose, periodic): # send and periodic options are mutually exclusive, if they both are # specified, exit. if send and periodic: print(_('--send and --periodic flags cannot be used together'), file=sys.stderr) exit(1) list_manager = getUtility(IListManager) if list_ids: lists = [] for spec in list_ids: # We'll accept list-ids or fqdn list names. if '@' in spec: mlist = list_manager.get(spec) else: mlist = list_manager.get_by_list_id(spec) if mlist is None: print(_('No such list found: $spec'), file=sys.stderr) else: lists.append(mlist) else: lists = list(list_manager.mailing_lists) if bump: for mlist in lists: if verbose: print(_('\ $mlist.list_id is at volume $mlist.volume, number \ ${mlist.next_digest_number}')) if not dry_run: bump_digest_number_and_volume(mlist) if verbose: print(_('\ $mlist.list_id bumped to volume $mlist.volume, number \ ${mlist.next_digest_number}')) if send: for mlist in lists: if verbose: print(_('\ $mlist.list_id sent volume $mlist.volume, number ${mlist.next_digest_number}')) if not dry_run: maybe_send_digest_now(mlist, force=True) if periodic: for mlist in lists: if mlist.digest_send_periodic: if verbose: print(_('\ $mlist.list_id sent volume $mlist.volume, number ${mlist.next_digest_number}')) if not dry_run: maybe_send_digest_now(mlist, force=True) @public @implementer(ICLISubCommand) class Digests: name = 'digests' command = digests mailman-3.2.2/src/mailman/commands/cli_help.py0000644000175000017500000000253613442110351022411 0ustar maxkingmaxking00000000000000# Copyright (C) 2009-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """The 'help' subcommand.""" import sys import click from mailman.core.i18n import _ from mailman.interfaces.command import ICLISubCommand from mailman.utilities.options import I18nCommand from public import public from zope.interface import implementer @click.command( cls=I18nCommand, help=_('Show this help message and exit.')) @click.pass_context # https://github.com/pallets/click/issues/832 def help(ctx): # pragma: nocover click.echo(ctx.parent.get_help(), color=ctx.color) sys.exit() @public @implementer(ICLISubCommand) class Help: name = 'help' command = help mailman-3.2.2/src/mailman/commands/cli_import.py0000644000175000017500000000572313442110351022774 0ustar maxkingmaxking00000000000000# Copyright (C) 2010-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Importing list data into Mailman 3.""" import sys import click import pickle from contextlib import ExitStack from mailman.core.i18n import _ from mailman.database.transaction import transaction from mailman.interfaces.command import ICLISubCommand from mailman.interfaces.listmanager import IListManager from mailman.utilities.importer import Import21Error, import_config_pck from mailman.utilities.modules import hacked_sys_modules from mailman.utilities.options import I18nCommand from public import public from zope.component import getUtility from zope.interface import implementer # A fake Bouncer class from Mailman 2.1, we don't use it but there are # instances in the .pck files. class Bouncer: class _BounceInfo: pass @click.command( cls=I18nCommand, help=_("""\ Import Mailman 2.1 list data'. Requires the fully-qualified name of the list to import and the path to the Mailman 2.1 pickle file.""")) @click.argument('listspec') @click.argument( 'pickle_file', metavar='PICKLE_FILE', type=click.File(mode='rb')) @click.pass_context def import21(ctx, listspec, pickle_file): mlist = getUtility(IListManager).get(listspec) if mlist is None: ctx.fail(_('No such list: $listspec')) with ExitStack() as resources: resources.enter_context(hacked_sys_modules('Mailman.Bouncer', Bouncer)) resources.enter_context(transaction()) while True: try: config_dict = pickle.load( pickle_file, encoding='utf-8', errors='ignore') except EOFError: break except pickle.UnpicklingError: ctx.fail( _('Not a Mailman 2.1 configuration file: $pickle_file')) else: if not isinstance(config_dict, dict): print(_('Ignoring non-dictionary: {0!r}').format( config_dict), file=sys.stderr) continue try: import_config_pck(mlist, config_dict) except Import21Error as error: print(error, file=sys.stderr) sys.exit(1) @public @implementer(ICLISubCommand) class Import21: name = 'import21' command = import21 mailman-3.2.2/src/mailman/commands/cli_info.py0000644000175000017500000000542013442110351022407 0ustar maxkingmaxking00000000000000# Copyright (C) 2009-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Information about this Mailman instance.""" import sys import click from lazr.config import as_boolean from mailman.config import config from mailman.core.api import API30, API31 from mailman.core.i18n import _ from mailman.interfaces.command import ICLISubCommand from mailman.utilities.options import I18nCommand from mailman.version import MAILMAN_VERSION_FULL from public import public from zope.interface import implementer @click.command( cls=I18nCommand, help=_('Information about this Mailman instance.')) @click.option( '--output', '-o', type=click.File(mode='w', encoding='utf-8', atomic=True), help=_("""\ File to send the output to. If not given, standard output is used.""")) @click.option( '--verbose', '-v', is_flag=True, default=False, help=_("""\ A more verbose output including the file system paths that Mailman is using.""")) def info(output, verbose): """See `ICLISubCommand`.""" print(MAILMAN_VERSION_FULL, file=output) print('Python', sys.version, file=output) print('config file:', config.filename, file=output) print('db url:', config.db.url, file=output) print('devmode:', 'ENABLED' if as_boolean(config.devmode.enabled) else 'DISABLED', file=output) api = (API30 if config.webservice.api_version == '3.0' else API31) print('REST root url:', api.path_to('/'), file=output) print('REST credentials: {}:{}'.format( config.webservice.admin_user, config.webservice.admin_pass), file=output) if verbose: print('File system paths:', file=output) longest = 0 paths = {} for attribute in dir(config): if attribute.endswith('_DIR') or attribute.endswith('_FILE'): paths[attribute] = getattr(config, attribute) longest = max(longest, len(attribute)) for attribute in sorted(paths): print(' {0:{2}} = {1}'.format( attribute, paths[attribute], longest)) @public @implementer(ICLISubCommand) class Info: name = 'info' command = info mailman-3.2.2/src/mailman/commands/cli_inject.py0000644000175000017500000000635513442110351022740 0ustar maxkingmaxking00000000000000# Copyright (C) 2009-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """The `mailman inject` subcommand.""" import sys import click from mailman.app.inject import inject_text from mailman.config import config from mailman.core.i18n import _ from mailman.interfaces.command import ICLISubCommand from mailman.interfaces.listmanager import IListManager from mailman.utilities.options import I18nCommand from public import public from zope.component import getUtility from zope.interface import implementer def show_queues(ctx, param, value): if value: print('Available queues:') for switchboard in sorted(config.switchboards): print(' ', switchboard) sys.exit(0) # Returning None tells click to process the rest of the command line. @click.command( cls=I18nCommand, help=_("Inject a message from a file into a mailing list's queue.")) @click.option( '--queue', '-q', help=_("""\ The name of the queue to inject the message to. QUEUE must be one of the directories inside the qfiles directory. If omitted, the incoming queue is used.""")) @click.option( '--show', '-s', is_flag=True, default=False, is_eager=True, expose_value=False, callback=show_queues, help=_('Show a list of all available queue names and exit.')) @click.option( '--filename', '-f', 'message_file', default='-', type=click.File(encoding='utf-8'), help=_("""\ Name of file containing the message to inject. If not given, or '-' (without the quotes) standard input is used.""")) @click.option( '--metadata', '-m', 'keywords', multiple=True, metavar='KEY=VALUE', help=_("""\ Additional metadata key/value pairs to add to the message metadata dictionary. Use the format key=value. Multiple -m options are allowed.""")) @click.argument('listspec') @click.pass_context def inject(ctx, queue, message_file, keywords, listspec): mlist = getUtility(IListManager).get(listspec) if mlist is None: ctx.fail(_('No such list: $listspec')) queue_name = ('in' if queue is None else queue) switchboard = config.switchboards.get(queue_name) if switchboard is None: ctx.fail(_('No such queue: $queue')) try: message_text = message_file.read() except KeyboardInterrupt: print('Interrupted') sys.exit(1) kws = {} for keyvalue in keywords: key, equals, value = keyvalue.partition('=') kws[key] = value inject_text(mlist, message_text, switchboard=queue, **kws) @public @implementer(ICLISubCommand) class Inject: name = 'inject' command = inject mailman-3.2.2/src/mailman/commands/cli_lists.py0000644000175000017500000002173113442110351022615 0ustar maxkingmaxking00000000000000# Copyright (C) 2009-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """The 'lists' subcommand.""" import sys import click from mailman.app.lifecycle import create_list, remove_list from mailman.core.constants import system_preferences from mailman.core.i18n import _ from mailman.database.transaction import transaction from mailman.email.message import UserNotification from mailman.interfaces.address import ( IEmailValidator, InvalidEmailAddressError) from mailman.interfaces.command import ICLISubCommand from mailman.interfaces.domain import ( BadDomainSpecificationError, IDomainManager) from mailman.interfaces.languages import ILanguageManager from mailman.interfaces.listmanager import IListManager, ListAlreadyExistsError from mailman.interfaces.template import ITemplateLoader from mailman.utilities.options import I18nCommand from mailman.utilities.string import expand, wrap from operator import attrgetter from public import public from zope.component import getUtility from zope.interface import implementer COMMASPACE = ', ' @click.command( cls=I18nCommand, help=_('List all mailing lists.')) @click.option( '--advertised', '-a', is_flag=True, default=False, help=_('List only those mailing lists that are publicly advertised')) @click.option( '--names/--no-names', '-n/-N', is_flag=True, default=False, help=_('Show also the list names')) @click.option( '--descriptions/--no-descriptions', '-d/-D', is_flag=True, default=False, help=_('Show also the list descriptions')) @click.option( '--quiet', '-q', is_flag=True, default=False, help=_('Less verbosity')) @click.option( '--domain', 'domains', multiple=True, metavar='DOMAIN', help=_("""\ List only those mailing lists hosted on the given domain, which must be the email host name. Multiple -d options may be given. """)) @click.pass_context def lists(ctx, advertised, names, descriptions, quiet, domains): mailing_lists = set() list_manager = getUtility(IListManager) # Gather the matching mailing lists. for mlist in list_manager.mailing_lists: if advertised and not mlist.advertised: continue if len(domains) > 0 and mlist.mail_host not in domains: continue mailing_lists.add(mlist) # Maybe no mailing lists matched. if len(mailing_lists) == 0: if not quiet: print(_('No matching mailing lists found')) sys.exit() count = len(mailing_lists) # noqa: F841 if not quiet: print(_('$count matching mailing lists found:')) # Calculate the longest identifier. longest = 0 output = [] for mlist in sorted(mailing_lists, key=attrgetter('list_id')): if names: identifier = '{} [{}]'.format( mlist.fqdn_listname, mlist.display_name) else: identifier = mlist.fqdn_listname longest = max(len(identifier), longest) output.append((identifier, mlist.description)) # Print it out. if descriptions: format_string = '{0:{2}} - {1:{3}}' else: format_string = '{0:{2}}' for identifier, description in output: print(format_string.format( identifier, description, longest, 70 - longest)) @public @implementer(ICLISubCommand) class Lists: name = 'lists' command = lists @click.command( cls=I18nCommand, help=_("""\ Create a mailing list. The 'fully qualified list name', i.e. the posting address of the mailing list is required. It must be a valid email address and the domain must be registered with Mailman. List names are forced to lower case.""")) @click.option( '--language', metavar='CODE', help=_("""\ Set the list's preferred language to CODE, which must be a registered two letter language code.""")) @click.option( '--owner', '-o', 'owners', multiple=True, metavar='OWNER', help=_("""\ Specify a list owner email address. If the address is not currently registered with Mailman, the address is registered and linked to a user. Mailman will send a confirmation message to the address, but it will also send a list creation notice to the address. More than one owner can be specified.""")) @click.option( '--notify/-no-notify', '-n/-N', default=False, help=_("""\ Notify the list owner by email that their mailing list has been created.""")) @click.option( '--quiet', '-q', is_flag=True, default=False, help=_('Print less output.')) @click.option( '--domain/--no-domain', '-d/-D', 'create_domain', default=True, help=_("""\ Register the mailing list's domain if not yet registered. This is the default behavior, but these options are provided for backward compatibility. With -D do not register the mailing list's domain.""")) @click.argument('fqdn_listname', metavar='LISTNAME') @click.pass_context def create(ctx, language, owners, notify, quiet, create_domain, fqdn_listname): language_code = (language if language is not None else system_preferences.preferred_language.code) # Make sure that the selected language code is known. if language_code not in getUtility(ILanguageManager).codes: ctx.fail(_('Invalid language code: $language_code')) # Check to see if the domain exists or not. listname, at, domain = fqdn_listname.partition('@') domain_manager = getUtility(IDomainManager) if domain_manager.get(domain) is None and create_domain: domain_manager.add(domain) # Validate the owner email addresses. The problem with doing this check in # create_list() is that you wouldn't be able to distinguish between an # InvalidEmailAddressError for the list name or the owners. I suppose we # could subclass that exception though. if len(owners) > 0: validator = getUtility(IEmailValidator) invalid_owners = [owner for owner in owners if not validator.is_valid(owner)] if invalid_owners: invalid = COMMASPACE.join(sorted(invalid_owners)) # noqa: F841 ctx.fail(_('Illegal owner addresses: $invalid')) try: mlist = create_list(fqdn_listname, owners) except InvalidEmailAddressError: ctx.fail(_('Illegal list name: $fqdn_listname')) except ListAlreadyExistsError: ctx.fail(_('List already exists: $fqdn_listname')) except BadDomainSpecificationError as domain: # noqa: F841 ctx.fail(_('Undefined domain: $domain')) # Find the language associated with the code, then set the mailing list's # preferred language to that. language_manager = getUtility(ILanguageManager) with transaction(): mlist.preferred_language = language_manager[language_code] # Do the notification. if not quiet: print(_('Created mailing list: $mlist.fqdn_listname')) if notify: template = getUtility(ITemplateLoader).get( 'domain:admin:notice:new-list', mlist) text = wrap(expand(template, mlist, dict( # For backward compatibility. requestaddr=mlist.request_address, siteowner=mlist.no_reply_address, ))) # Set the I18N language to the list's preferred language so the header # will match the template language. Stashing and restoring the old # translation context is just (healthy? :) paranoia. with _.using(mlist.preferred_language.code): msg = UserNotification( owners, mlist.no_reply_address, _('Your new mailing list: $fqdn_listname'), text, mlist.preferred_language) msg.send(mlist) @public @implementer(ICLISubCommand) class Create: name = 'create' command = create @click.command( cls=I18nCommand, help=_('Remove a mailing list.')) @click.option( '--quiet', '-q', is_flag=True, default=False, help=_('Suppress status messages')) @click.argument('listspec') def remove(quiet, listspec): mlist = getUtility(IListManager).get(listspec) if mlist is None: if not quiet: print(_('No such list matching spec: $listspec')) sys.exit(0) with transaction(): remove_list(mlist) if not quiet: print(_('Removed list: $listspec')) @public @implementer(ICLISubCommand) class Remove: name = 'remove' command = remove mailman-3.2.2/src/mailman/commands/cli_members.py0000644000175000017500000001701013442110351023104 0ustar maxkingmaxking00000000000000# Copyright (C) 2009-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """The 'members' subcommand.""" import click from email.utils import formataddr, parseaddr from mailman.app.membership import add_member from mailman.core.i18n import _ from mailman.database.transaction import transactional from mailman.interfaces.command import ICLISubCommand from mailman.interfaces.listmanager import IListManager from mailman.interfaces.member import ( AlreadySubscribedError, DeliveryMode, DeliveryStatus, MemberRole) from mailman.interfaces.subscriptions import RequestRecord from mailman.utilities.options import I18nCommand from operator import attrgetter from public import public from zope.component import getUtility from zope.interface import implementer def display_members(ctx, mlist, role, regular, digest, nomail, outfp): # Which type of digest recipients should we display? if digest == 'any': digest_types = [ DeliveryMode.plaintext_digests, DeliveryMode.mime_digests, DeliveryMode.summary_digests, ] elif digest is not None: digest_types = [DeliveryMode[digest + '_digests']] else: # Don't filter on digest type. pass # Which members with delivery disabled should we display? if nomail is None: # Don't filter on delivery status. pass elif nomail == 'byadmin': status_types = [DeliveryStatus.by_moderator] elif nomail.startswith('by'): status_types = [DeliveryStatus['by_' + nomail[2:]]] elif nomail == 'enabled': status_types = [DeliveryStatus.enabled] elif nomail == 'unknown': status_types = [DeliveryStatus.unknown] elif nomail == 'any': status_types = [ DeliveryStatus.by_user, DeliveryStatus.by_bounces, DeliveryStatus.by_moderator, DeliveryStatus.unknown, ] else: # pragma: nocover # click should enforce a valid nomail option. raise AssertionError(nomail) # Which roles should we display? if role is None: # By default, filter on members. roster = mlist.members elif role == 'administrator': roster = mlist.administrators elif role == 'any': roster = mlist.subscribers else: # click should enforce a valid member role. roster = mlist.get_roster(MemberRole[role]) # Print; outfp will be either the file or stdout to print to. addresses = list(roster.addresses) if len(addresses) == 0: print(_('$mlist.list_id has no members'), file=outfp) return for address in sorted(addresses, key=attrgetter('email')): if regular: member = roster.get_member(address.email) if member.delivery_mode != DeliveryMode.regular: continue if digest is not None: member = roster.get_member(address.email) if member.delivery_mode not in digest_types: continue if nomail is not None: member = roster.get_member(address.email) if member.delivery_status not in status_types: continue print(formataddr((address.display_name, address.original_email)), file=outfp) @transactional def add_members(mlist, infp): for line in infp: # Ignore blank lines and lines that start with a '#'. if line.startswith('#') or len(line.strip()) == 0: continue # Parse the line and ensure that the values are unicodes. display_name, email = parseaddr(line) try: add_member(mlist, RequestRecord(email, display_name, DeliveryMode.regular, mlist.preferred_language.code)) except AlreadySubscribedError: # It's okay if the address is already subscribed, just print a # warning and continue. if not display_name: print(_('Already subscribed (skipping): $email')) else: print(_('Already subscribed (skipping): ' '$display_name <$email>')) @click.command( cls=I18nCommand, help=_("""\ Display a mailing list's members, with filtering along various criteria. """)) @click.option( '--add', '-a', 'infp', metavar='FILENAME', type=click.File(encoding='utf-8'), help=_("""\ Add all member addresses in FILENAME. FILENAME can be '-' to indicate standard input. Blank lines and lines That start with a '#' are ignored. Without this option, this command displays mailing list members.""")) @click.option( '--output', '-o', 'outfp', metavar='FILENAME', type=click.File(mode='w', encoding='utf-8', atomic=True), help=_("""Display output to FILENAME instead of stdout. FILENAME can be '-' to indicate standard output.""")) @click.option( '--role', '-R', type=click.Choice(('any', 'owner', 'moderator', 'nonmember', 'member', 'administrator')), help=_("""\ Display only members with a given ROLE. The role may be 'any', 'member', 'nonmember', 'owner', 'moderator', or 'administrator' (i.e. owners and moderators). If not given, then delivery members are used. """)) @click.option( '--regular', '-r', is_flag=True, default=False, help=_('Display only regular delivery members.')) @click.option( '--digest', '-d', metavar='kind', # baw 2010-01-23 summary digests are not really supported yet. type=click.Choice(('any', 'plaintext', 'mime')), help=_("""\ Display only digest members of kind. 'any' means any digest type, 'plaintext' means only plain text (rfc 1153) type digests, 'mime' means mime type digests.""")) @click.option( '--nomail', '-n', metavar='WHY', type=click.Choice(('enabled', 'any', 'unknown', 'byadmin', 'byuser', 'bybounces')), help=_("""\ Display only members with a given delivery status. 'enabled' means all members whose delivery is enabled, 'any' means members whose delivery is disabled for any reason, 'byuser' means that the member disabled their own delivery, 'bybounces' means that delivery was disabled by the automated bounce processor, 'byadmin' means delivery was disabled by the list administrator or moderator, and 'unknown' means that delivery was disabled for unknown (legacy) reasons.""")) @click.argument('listspec') @click.pass_context def members(ctx, infp, outfp, role, regular, digest, nomail, listspec): mlist = getUtility(IListManager).get(listspec) if mlist is None: ctx.fail(_('No such list: $listspec')) if infp is None: display_members(ctx, mlist, role, regular, digest, nomail, outfp) else: add_members(mlist, infp) @public @implementer(ICLISubCommand) class Members: name = 'members' command = members mailman-3.2.2/src/mailman/commands/cli_qfile.py0000644000175000017500000000556013442110351022561 0ustar maxkingmaxking00000000000000# Copyright (C) 2009-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Getting information out of a qfile.""" import click import pickle from mailman.core.i18n import _ from mailman.interfaces.command import ICLISubCommand from mailman.utilities.interact import interact from mailman.utilities.options import I18nCommand from pprint import PrettyPrinter from public import public from zope.interface import implementer # This is deliberately called 'm' for use with --interactive. m = None @click.command( cls=I18nCommand, help=_('Get information out of a queue file.')) @click.option( '--print/--no-print', '-p/-n', 'doprint', default=True, help=_("""\ Don't attempt to pretty print the object. This is useful if there is some problem with the object and you just want to get an unpickled representation. Useful with 'mailman qfile -i '. In that case, the list of unpickled objects will be left in a variable called 'm'.""")) @click.option( '--interactive', '-i', is_flag=True, default=False, help=_("""\ Start an interactive Python session, with a variable called 'm' containing the list of unpickled objects.""")) @click.argument('qfile') def qfile(doprint, interactive, qfile): global m # Reinitialize 'm' every time this command is run. This isn't normally # needed for command line use, but is important for the test suite. m = [] printer = PrettyPrinter(indent=4) with open(qfile, 'rb') as fp: while True: try: m.append(pickle.load(fp)) except EOFError: break if doprint: print(_('[----- start pickle -----]')) for i, obj in enumerate(m): count = i + 1 print(_('<----- start object $count ----->')) if isinstance(obj, (bytes, str)): print(obj) else: printer.pprint(obj) print(_('[----- end pickle -----]')) count = len(m) # noqa: F841 banner = _("Number of objects found (see the variable 'm'): $count") if interactive: interact(banner=banner) @public @implementer(ICLISubCommand) class QFile: name = 'qfile' command = qfile mailman-3.2.2/src/mailman/commands/cli_status.py0000644000175000017500000000407613442110351023005 0ustar maxkingmaxking00000000000000# Copyright (C) 2010-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """The `mailman status` subcommand.""" import sys import click import socket from mailman.bin.master import WatcherState, master_state from mailman.core.i18n import _ from mailman.interfaces.command import ICLISubCommand from mailman.utilities.options import I18nCommand from public import public from zope.interface import implementer @click.command( cls=I18nCommand, help=_('Show the current running status of the Mailman system.')) def status(): status, lock = master_state() if status is WatcherState.none: message = _('GNU Mailman is not running') elif status is WatcherState.conflict: hostname, pid, tempfile = lock.details message = _('GNU Mailman is running (master pid: $pid)') elif status is WatcherState.stale_lock: hostname, pid, tempfile = lock.details message = _('GNU Mailman is stopped (stale pid: $pid)') else: hostname, pid, tempfile = lock.details fqdn_name = socket.getfqdn() # noqa: F841 assert status is WatcherState.host_mismatch, ( 'Invalid enum value: %s' % status) message = _('GNU Mailman is in an unexpected state ' '($hostname != $fqdn_name)') print(message) sys.exit(status.value) @public @implementer(ICLISubCommand) class Status: name = 'status' command = status mailman-3.2.2/src/mailman/commands/cli_unshunt.py0000644000175000017500000000377313442110351023171 0ustar maxkingmaxking00000000000000# Copyright (C) 2009-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """The 'unshunt' command.""" import sys import click from mailman.config import config from mailman.core.i18n import _ from mailman.interfaces.command import ICLISubCommand from mailman.utilities.options import I18nCommand from public import public from zope.interface import implementer @click.command( cls=I18nCommand, help=_('Unshunt messages.')) @click.option( '--discard', '-d', is_flag=True, default=False, help=_("""\ Discard all shunted messages instead of moving them back to their original queue.""")) def unshunt(discard): shunt_queue = config.switchboards['shunt'] shunt_queue.recover_backup_files() for filebase in shunt_queue.files: try: msg, msgdata = shunt_queue.dequeue(filebase) which_queue = msgdata.get('whichq', 'in') if not discard: config.switchboards[which_queue].enqueue(msg, msgdata) except Exception as error: # noqa: F841 print(_('Cannot unshunt message $filebase, skipping:\n$error'), file=sys.stderr) else: # Unlink the .bak file left by dequeue() shunt_queue.finish(filebase) @public @implementer(ICLISubCommand) class Unshunt: name = 'unshunt' command = unshunt mailman-3.2.2/src/mailman/commands/cli_version.py0000644000175000017500000000234613442110351023145 0ustar maxkingmaxking00000000000000# Copyright (C) 2009-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """The Mailman version.""" import click from mailman.core.i18n import _ from mailman.interfaces.command import ICLISubCommand from mailman.utilities.options import I18nCommand from mailman.version import MAILMAN_VERSION_FULL from public import public from zope.interface import implementer @click.command( cls=I18nCommand, help=_("Display Mailman's version.")) def version(): print(MAILMAN_VERSION_FULL) @public @implementer(ICLISubCommand) class Version: name = 'version' command = version mailman-3.2.2/src/mailman/commands/cli_withlist.py0000644000175000017500000002561413442110351023332 0ustar maxkingmaxking00000000000000# Copyright (C) 2009-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """The `mailman shell` subcommand.""" import re import sys import click from contextlib import ExitStack, suppress from functools import partial from lazr.config import as_boolean from mailman.config import config from mailman.core.i18n import _ from mailman.interfaces.command import ICLISubCommand from mailman.interfaces.listmanager import IListManager from mailman.utilities.interact import DEFAULT_BANNER, interact from mailman.utilities.modules import call_name from mailman.utilities.options import I18nCommand from public import public from string import Template from traceback import print_exc from zope.component import getUtility from zope.interface import implementer # Global holding onto the open mailing list. m = None # Global holding the results of --run. r = None def start_ipython1(overrides, banner, *, debug=False): try: from IPython.frontend.terminal.embed import InteractiveShellEmbed except ImportError: if debug: print_exc() return None return InteractiveShellEmbed.instance(banner1=banner, user_ns=overrides) def start_ipython4(overrides, banner, *, debug=False): try: from IPython.terminal.embed import InteractiveShellEmbed shell = InteractiveShellEmbed.instance() except ImportError: if debug: print_exc() return None return partial(shell.mainloop, local_ns=overrides, display_banner=banner) def start_ipython(overrides, banner, debug): shell = None for starter in (start_ipython4, start_ipython1): shell = starter(overrides, banner, debug=debug) if shell is not None: shell() break else: print(_('ipython is not available, set use_ipython to no')) def start_python(overrides, banner): # Set the tab completion. with ExitStack() as resources: try: # pragma: nocover import readline, rlcompleter # noqa: F401,E401 except ImportError: # pragma: nocover print(_('readline not available'), file=sys.stderr) pass else: readline.parse_and_bind('tab: complete') history_file_template = config.shell.history_file.strip() if len(history_file_template) > 0: # Expand substitutions. substitutions = { key.lower(): value for key, value in config.paths.items() } history_file = Template( history_file_template).safe_substitute(substitutions) with suppress(FileNotFoundError): readline.read_history_file(history_file) resources.callback( readline.write_history_file, history_file) sys.ps1 = config.shell.prompt + ' ' interact(upframe=False, banner=banner, overrides=overrides) def do_interactive(ctx, banner): global m, r overrides = dict( m=m, commit=config.db.commit, abort=config.db.abort, config=config, getUtility=getUtility ) # Bootstrap some useful names into the namespace, mostly to make # the component architecture and interfaces easily available. for module_name in sys.modules: if not module_name.startswith('mailman.interfaces.'): continue module = sys.modules[module_name] for name in module.__all__: overrides[name] = getattr(module, name) banner = config.shell.banner + '\n' + ( banner if isinstance(banner, str) else '') try: use_ipython = as_boolean(config.shell.use_ipython) except ValueError: if config.shell.use_ipython == 'debug': use_ipython = True debug = True else: print(_('Invalid value for [shell]use_python: {}').format( config.shell.use_ipython), file=sys.stderr) return else: debug = False if use_ipython: start_ipython(overrides, banner, debug) else: start_python(overrides, banner) def show_detailed_help(ctx, param, value): if not value: # Returning None tells click to process the rest of the command line. return # Split this up into paragraphs for easier translation. print(_("""\ This script provides you with a general framework for interacting with a mailing list.""")) print() print(_("""\ There are two ways to use this script: interactively or programmatically. Using it interactively allows you to play with, examine and modify a mailing list from Python's interactive interpreter. When running interactively, the variable 'm' will be available in the global namespace. It will reference the mailing list object.""")) print() print(_("""\ Programmatically, you can write a function to operate on a mailing list, and this script will take care of the housekeeping (see below for examples). In that case, the general usage syntax is: % mailman withlist [options] -l listspec [args ...] where `listspec` is either the posting address of the mailing list (e.g. ant@example.com), or the List-ID (e.g. ant.example.com).""")) print() print(_("""\ Here's an example of how to use the --run option. Say you have a file in the Mailman installation directory called 'listaddr.py', with the following two functions: def listaddr(mlist): print(mlist.posting_address) def requestaddr(mlist): print(mlist.request_address) Run methods take at least one argument, the mailing list object to operate on. Any additional arguments given on the command line are passed as positional arguments to the callable. If -l is not given then you can run a function that takes no arguments. """)) print() print(_("""\ You can print the list's posting address by running the following from the command line: % mailman withlist -r listaddr -l ant@example.com Importing listaddr ... Running listaddr.listaddr() ... ant@example.com""")) print() print(_("""\ And you can print the list's request address by running: % mailman withlist -r listaddr.requestaddr -l ant@example.com Importing listaddr ... Running listaddr.requestaddr() ... ant-request@example.com""")) print() print(_("""\ As another example, say you wanted to change the display name for a particular mailing list. You could put the following function in a file called `change.py`: def change(mlist, display_name): mlist.display_name = display_name and run this from the command line: % mailman withlist -r change -l ant@example.com 'My List' Note that you do not have to explicitly commit any database transactions, as Mailman will do this for you (assuming no errors occured).""")) sys.exit(0) @click.command( cls=I18nCommand, help=_("""\ Operate on a mailing list. For detailed help, see --details """)) @click.option( '--interactive', '-i', is_flag=True, default=None, help=_("""\ Leaves you at an interactive prompt after all other processing is complete. This is the default unless the --run option is given.""")) @click.option( '--run', '-r', help=_("""\ Run a script. The argument is the module path to a callable. This callable will be imported and then, if --listspec/-l is also given, is called with the mailing list as the first argument. If additional arguments are given at the end of the command line, they are passed as subsequent positional arguments to the callable. For additional help, see --details. If no --listspec/-l argument is given, the script function being called is called with no arguments. """)) @click.option( '--details', is_flag=True, default=False, is_eager=True, expose_value=False, callback=show_detailed_help, help=_('Print detailed instructions and exit.')) # Optional positional argument. @click.option( '--listspec', '-l', help=_("""\ A specification of the mailing list to operate on. This may be the posting address of the list, or its List-ID. The argument can also be a Python regular expression, in which case it is matched against both the posting address and List-ID of all mailing lists. To use a regular expression, LISTSPEC must start with a ^ (and the matching is done with re.match(). LISTSPEC cannot be a regular expression unless --run is given.""")) @click.argument('run_args', nargs=-1) @click.pass_context def shell(ctx, interactive, run, listspec, run_args): global m, r banner = DEFAULT_BANNER # Interactive is the default unless --run was given. interactive = (run is None) if interactive is None else interactive # List name cannot be a regular expression if --run is not given. if listspec and listspec.startswith('^') and not run: ctx.fail(_('Regular expression requires --run')) # Handle --run. list_manager = getUtility(IListManager) if run: # When the module and the callable have the same name, a shorthand # without the dot is allowed. dotted_name = (run if '.' in run else '{0}.{0}'.format(run)) if listspec is None: r = call_name(dotted_name, *run_args) elif listspec.startswith('^'): r = {} cre = re.compile(listspec, re.IGNORECASE) for mlist in list_manager.mailing_lists: if cre.match(mlist.fqdn_listname) or cre.match(mlist.list_id): results = call_name(dotted_name, mlist, *run_args) r[mlist.list_id] = results else: m = list_manager.get(listspec) if m is None: ctx.fail(_('No such list: $listspec')) r = call_name(dotted_name, m, *run_args) else: # Not --run. if listspec is not None: m = list_manager.get(listspec) if m is None: ctx.fail(_('No such list: $listspec')) banner = _("The variable 'm' is the $listspec mailing list") # All other processing is finished; maybe go into interactive mode. if interactive: do_interactive(ctx, banner) @public @implementer(ICLISubCommand) class Withlist: name = 'withlist' command = shell @public class Shell(Withlist): name = 'shell' command = shell mailman-3.2.2/src/mailman/commands/docs/0000755000175000017500000000000013445614541021217 5ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/commands/docs/__init__.py0000644000175000017500000000000013244427337023320 0ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/commands/docs/aliases.rst0000644000175000017500000000521613421245331023365 0ustar maxkingmaxking00000000000000================== Generating aliases ================== For some mail servers, Mailman must generate data files that are used to hook Mailman up to the mail server. The details of this differ for each mail server. Generally these files are automatically kept up-to-date when mailing lists are created or removed, but you might occasionally need to manually regenerate the file. The ``mailman aliases`` command does this. >>> command = cli('mailman.commands.cli_aliases.aliases') For example, connecting Mailman to Postfix is generally done through the LMTP protocol. Mailman starts an LMTP server and Postfix delivers messages to Mailman as an LMTP client. By default this is done through Postfix transport maps. Selecting Postfix as the source of incoming messages enables transport map generation. >>> config.push('postfix', """ ... [mta] ... incoming: mailman.mta.postfix.LMTP ... lmtp_host: lmtp.example.com ... lmtp_port: 24 ... """) .. Clean up. >>> ignore = cleanups.callback(config.pop, 'postfix') Let's create a mailing list and then display the transport map for it. We'll write the appropriate files to a temporary directory. :: >>> mlist = create_list('ant@example.com') >>> import os, shutil, tempfile >>> output_directory = tempfile.mkdtemp() >>> ignore = cleanups.callback(shutil.rmtree, output_directory) >>> command('mailman aliases --directory ' + output_directory) For Postfix, there are two files in the output directory. >>> files = sorted(os.listdir(output_directory)) >>> for file in files: ... print(file) postfix_domains postfix_lmtp The transport map file contains all the aliases for the mailing list. >>> with open(os.path.join(output_directory, 'postfix_lmtp')) as fp: ... print(fp.read()) # AUTOMATICALLY GENERATED BY MAILMAN ON ... ... ant@example.com lmtp:[lmtp.example.com]:24 ant-bounces@example.com lmtp:[lmtp.example.com]:24 ant-confirm@example.com lmtp:[lmtp.example.com]:24 ant-join@example.com lmtp:[lmtp.example.com]:24 ant-leave@example.com lmtp:[lmtp.example.com]:24 ant-owner@example.com lmtp:[lmtp.example.com]:24 ant-request@example.com lmtp:[lmtp.example.com]:24 ant-subscribe@example.com lmtp:[lmtp.example.com]:24 ant-unsubscribe@example.com lmtp:[lmtp.example.com]:24 The relay domains file contains a list of all the domains. >>> with open(os.path.join(output_directory, 'postfix_domains')) as fp: ... print(fp.read()) # AUTOMATICALLY GENERATED BY MAILMAN ON ... ... example.com example.com mailman-3.2.2/src/mailman/commands/docs/commands.rst0000644000175000017500000000007313244427337023554 0ustar maxkingmaxking00000000000000======== Commands ======== .. toctree:: :glob: ./* mailman-3.2.2/src/mailman/commands/docs/conf.rst0000644000175000017500000000402613421245331022667 0ustar maxkingmaxking00000000000000============================ Display configuration values ============================ Just like the `Postfix command postconf(1)`_, the ``mailman conf`` command lets you dump one or more Mailman configuration variables to standard output or a file. Mailman's configuration is divided in multiple sections which contain multiple key-value pairs. The ``mailman conf`` command allows you to display a specific key-value pair, or several key-value pairs. >>> command = cli('mailman.commands.cli_conf.conf') To get a list of all key-value pairs of any section, you need to call the command without any options. >>> command('mailman conf') [antispam] header_checks: ... [logging.bounce] level: info ... [mailman] site_owner: noreply@example.com ... You can list all the key-value pairs of a specific section. >>> command('mailman conf --section shell') [shell] banner: Welcome to the GNU Mailman shell [shell] history_file: [shell] prompt: >>> [shell] use_ipython: no You can also pass a key and display all key-value pairs matching the given key, along with the names of the corresponding sections. >>> command('mailman conf --key path') [logging.archiver] path: mailman.log [logging.bounce] path: bounce.log [logging.config] path: mailman.log [logging.database] path: mailman.log [logging.debug] path: debug.log [logging.error] path: mailman.log [logging.fromusenet] path: mailman.log [logging.http] path: mailman.log [logging.locks] path: mailman.log [logging.mischief] path: mailman.log [logging.plugins] path: plugins.log [logging.root] path: mailman.log [logging.runner] path: mailman.log [logging.smtp] path: smtp.log [logging.subscribe] path: mailman.log [logging.vette] path: mailman.log If you specify both a section and a key, you will get the corresponding value. >>> command('mailman conf --section mailman --key site_owner') noreply@example.com .. _`Postfix command postconf(1)`: http://www.postfix.org/postconf.1.html mailman-3.2.2/src/mailman/commands/docs/control.rst0000644000175000017500000000261613442110351023421 0ustar maxkingmaxking00000000000000============================= Starting and stopping Mailman ============================= The Mailman daemon processes can be started and stopped from the command line. Set up ====== All we care about is the master process; normally it starts a bunch of runners, but we don't care about any of them, so write a test configuration file for the master that disables all the runners. >>> from mailman.commands.tests.test_cli_control import make_config >>> make_config(cleanups) Starting ======== >>> command = cli('mailman.commands.cli_control.start') Starting the daemons prints a useful message and starts the master watcher process in the background. >>> command('mailman start') Starting Mailman's master runner >>> from mailman.commands.tests.test_cli_control import find_master The process exists, and its pid is available in a run time file. >>> pid = find_master() >>> pid is not None True Stopping ======== You can also stop the master watcher process from the command line, which stops all the child processes too. :: >>> command = cli('mailman.commands.cli_control.stop') >>> command('mailman stop') Shutting down Mailman's master runner .. # Clean up. >>> from mailman.commands.tests.test_cli_control import ( ... kill_with_extreme_prejudice, clean_stale_locks) >>> kill_with_extreme_prejudice(pid) >>> clean_stale_locks() mailman-3.2.2/src/mailman/commands/docs/create.rst0000644000175000017500000001067713421245331023216 0ustar maxkingmaxking00000000000000========================== Command line list creation ========================== A system administrator can create mailing lists by the command line. >>> command = cli('mailman.commands.cli_lists.create') You can prevent creation of a mailing list in an unknown domain. >>> command('mailman create --no-domain ant@example.xx') Usage: create [OPTIONS] LISTNAME Try "create --help" for help. Error: Undefined domain: example.xx By default, Mailman will create the domain if it doesn't exist. >>> command('mailman create ant@example.xx') Created mailing list: ant@example.xx Now both the domain and the mailing list exist in the database. :: >>> from mailman.interfaces.listmanager import IListManager >>> from zope.component import getUtility >>> list_manager = getUtility(IListManager) >>> list_manager.get('ant@example.xx') >>> from mailman.interfaces.domain import IDomainManager >>> getUtility(IDomainManager).get('example.xx') The command can also operate quietly. :: >>> command('mailman create --quiet bee@example.com') >>> mlist = list_manager.get('bee@example.com') >>> mlist Setting the owner ================= By default, no list owners are specified. >>> dump_list(mlist.owners.addresses) *Empty* But you can specify an owner address on the command line when you create the mailing list. :: >>> command('mailman create --owner anne@example.com cat@example.com') Created mailing list: cat@example.com >>> mlist = list_manager.get('cat@example.com') >>> dump_list(repr(address) for address in mlist.owners.addresses) You can even specify more than one address for the owners. :: >>> command('mailman create ' ... '--owner anne@example.com ' ... '--owner bart@example.com ' ... '--owner cate@example.com ' ... 'dog@example.com') Created mailing list: dog@example.com >>> mlist = list_manager.get('dog@example.com') >>> from operator import attrgetter >>> dump_list(repr(address) for address in mlist.owners.addresses) Setting the language ==================== You can set the default language for the new mailing list when you create it. The language must be known to Mailman. :: >>> command('mailman create --language xx ewe@example.com') Usage: create [OPTIONS] LISTNAME Try "create --help" for help. Error: Invalid language code: xx >>> from mailman.interfaces.languages import ILanguageManager >>> getUtility(ILanguageManager).add('xx', 'iso-8859-1', 'Freedonian') >>> command('mailman create --language xx ewe@example.com') Created mailing list: ewe@example.com >>> mlist = list_manager.get('ewe@example.com') >>> print(mlist.preferred_language) Notifications ============= When told to, Mailman will notify the list owners of their new mailing list. >>> command('mailman create ' ... '--notify ' ... '--owner anne@example.com ' ... '--owner bart@example.com ' ... '--owner cate@example.com ' ... 'fly@example.com') Created mailing list: fly@example.com The notification message is in the virgin queue. :: >>> from mailman.testing.helpers import get_queue_messages >>> messages = get_queue_messages('virgin') >>> len(messages) 1 >>> for message in messages: ... print(message.msg.as_string()) MIME-Version: 1.0 ... Subject: Your new mailing list: fly@example.com From: noreply@example.com To: anne@example.com, bart@example.com, cate@example.com ... The mailing list 'fly@example.com' has just been created for you. The following is some basic information about your mailing list. There is an email-based interface for users (not administrators) of your list; you can get info about using it by sending a message with just the word 'help' as subject or in the body, to: fly-request@example.com Please address all questions to noreply@example.com. mailman-3.2.2/src/mailman/commands/docs/echo.rst0000644000175000017500000000152613244427337022675 0ustar maxkingmaxking00000000000000================== The 'echo' command ================== The mail command 'echo' simply replies with the original command and arguments to the sender. >>> command = config.commands['echo'] >>> print(command.name) echo >>> print(command.argument_description) [args] >>> print(command.description) Echo back your arguments. The original message is ignored, but the results receive the echoed command. :: >>> mlist = create_list('test@example.com') >>> from mailman.runners.command import Results >>> results = Results() >>> from mailman.email.message import Message >>> print(command.process(mlist, Message(), {}, ('foo', 'bar'), results)) ContinueProcessing.yes >>> print(str(results)) The results of your email command are provided below. echo foo bar mailman-3.2.2/src/mailman/commands/docs/end.rst0000644000175000017500000000205313244427337022521 0ustar maxkingmaxking00000000000000================= The 'end' command ================= The mail command processor recognized an 'end' command which tells it to stop processing email messages. >>> command = config.commands['end'] >>> print(command.name) end >>> print(command.description) Stop processing commands. The 'end' command takes no arguments. >>> print('DESCRIPTION:', command.argument_description) DESCRIPTION: The command itself is fairly simple; it just stops command processing, and the message isn't even looked at. >>> mlist = create_list('test@example.com') >>> from mailman.email.message import Message >>> print(command.process(mlist, Message(), {}, (), None)) ContinueProcessing.no The 'stop' command is a synonym for 'end'. >>> command = config.commands['stop'] >>> print(command.name) stop >>> print(command.description) An alias for 'end'. >>> print('DESCRIPTION:', command.argument_description) DESCRIPTION: >>> print(command.process(mlist, Message(), {}, (), None)) ContinueProcessing.no mailman-3.2.2/src/mailman/commands/docs/help.rst0000644000175000017500000000450313244427337022705 0ustar maxkingmaxking00000000000000================== Email command help ================== You can get some help about the various email commands that are available by sending the word `help` to a mailing list's -request address. >>> mlist = create_list('test@example.com') >>> from mailman.commands.eml_help import Help >>> help = Help() >>> print(help.name) help >>> print(help.description) Get help about available email commands. >>> print(help.argument_description) [command] With no arguments, `help` provides a list of the available commands and a short description of each of them. :: >>> from mailman.runners.command import Results >>> results = Results() >>> from mailman.email.message import Message >>> print(help.process(mlist, Message(), {}, (), results)) ContinueProcessing.yes >>> print(results) The results of your email command are provided below. confirm - Confirm a subscription request. echo - Echo back your arguments. end - Stop processing commands. help - Get help about available email commands. join - Join this mailing list. leave - Leave this mailing list. stop - An alias for 'end'. subscribe - An alias for 'join'. unsubscribe - An alias for 'leave'. With an argument, you can get more detailed help about a specific command. >>> results = Results() >>> print(help.process(mlist, Message(), {}, ('help',), results)) ContinueProcessing.yes >>> print(results) The results of your email command are provided below. help [command] Get help about available email commands. Some commands have even more detailed help. >>> results = Results() >>> print(help.process(mlist, Message(), {}, ('join',), results)) ContinueProcessing.yes >>> print(results) The results of your email command are provided below. join [digest=] Join this mailing list. You will be asked to confirm your subscription request and you may be issued a provisional password. By using the 'digest' option, you can specify whether you want digest delivery or not. If not specified, the mailing list's default delivery mode will be used. mailman-3.2.2/src/mailman/commands/docs/import.rst0000644000175000017500000000347113432413333023260 0ustar maxkingmaxking00000000000000=================== Importing list data =================== If you have the ``config.pck`` file for a version 2.1 mailing list, you can import that into an existing mailing list in Mailman 3.0. >>> command = cli('mailman.commands.cli_import.import21') You must specify the mailing list you are importing into, and it must exist. >>> command('mailman import21') Usage: ... [OPTIONS] LISTSPEC PICKLE_FILE Try "import21 --help" for help. Error: Missing argument "LISTSPEC". You must also specify a pickle file to import. >>> command('mailman import21 import@example.com') Usage: ... [OPTIONS] LISTSPEC PICKLE_FILE Try "import21 --help" for help. Error: Missing argument "PICKLE_FILE". Too bad the list doesn't exist. >>> from importlib_resources import path >>> with path('mailman.testing', 'config.pck') as pickle_path: ... pickle_file = str(pickle_path) ... command('mailman import21 import@example.com ' + pickle_file) Usage: ... [OPTIONS] LISTSPEC PICKLE_FILE Try "import21 --help" for help. Error: No such list: import@example.com When the mailing list exists, you must specify a real pickle file to import from. :: >>> mlist = create_list('import@example.com') >>> transaction.commit() >>> command('mailman import21 import@example.com ' + __file__) Usage: ... [OPTIONS] LISTSPEC PICKLE_FILE Try "import21 --help" for help. Error: Not a Mailman 2.1 configuration file: .../import.rst'... Now we can import the test pickle file. As a simple illustration of the import, the mailing list's "real name" will change. :: >>> print(mlist.display_name) Import >>> command('mailman import21 import@example.com ' + pickle_file) >>> print(mlist.display_name) Test mailman-3.2.2/src/mailman/commands/docs/info.rst0000644000175000017500000000434113421245331022675 0ustar maxkingmaxking00000000000000=================== Getting information =================== You can get information about Mailman's environment by using the command line script ``mailman info``. By default, the info is printed to standard output. :: >>> command = cli('mailman.commands.cli_info.info') >>> command('mailman info') GNU Mailman 3... Python ... ... config file: .../test.cfg db url: ... REST root url: http://localhost:9001/3.1/ REST credentials: restadmin:restpass By passing in the ``-o/--output`` option, you can print the info to a file. >>> from mailman.config import config >>> import os >>> output_path = os.path.join(config.VAR_DIR, 'output.txt') >>> command('mailman info -o ' + output_path) >>> with open(output_path) as fp: ... print(fp.read()) GNU Mailman 3... Python ... ... config file: .../test.cfg db url: ... devmode: DISABLED REST root url: http://localhost:9001/3.1/ REST credentials: restadmin:restpass You can also get more verbose information, which contains a list of the file system paths that Mailman is using. >>> config.create_paths = False >>> config.push('fhs', """ ... [mailman] ... layout: fhs ... """) >>> ignore = cleanups.callback(config.pop, 'fhs') >>> config.create_paths = True The `Filesystem Hierarchy Standard`_ layout is the same everywhere by definition. >>> command('mailman info --verbose') GNU Mailman 3... Python ... ... File system paths: ARCHIVE_DIR = /var/lib/mailman/archives BIN_DIR = /sbin CACHE_DIR = /var/lib/mailman/cache CFG_FILE = .../test.cfg DATA_DIR = /var/lib/mailman/data ETC_DIR = /etc LIST_DATA_DIR = /var/lib/mailman/lists LOCK_DIR = /var/lock/mailman LOCK_FILE = /var/lock/mailman/master.lck LOG_DIR = /var/log/mailman MESSAGES_DIR = /var/lib/mailman/messages PID_FILE = /var/run/mailman/master.pid QUEUE_DIR = /var/spool/mailman TEMPLATE_DIR = .../mailman/templates VAR_DIR = /var/lib/mailman .. _`Filesystem Hierarchy Standard`: http://www.pathname.com/fhs/ mailman-3.2.2/src/mailman/commands/docs/inject.rst0000644000175000017500000000742613421245331023225 0ustar maxkingmaxking00000000000000============================== Command line message injection ============================== You can inject a message directly into a queue directory via the command line. >>> command = cli('mailman.commands.cli_inject.inject') It's easy to find out which queues are available. >>> command('mailman inject --show') Available queues: archive bad bounces command digest in nntp out pipeline retry shunt virgin Usually, the text of the message to inject is in a file. >>> from tempfile import NamedTemporaryFile >>> filename = cleanups.enter_context(NamedTemporaryFile()).name >>> with open(filename, 'w', encoding='utf-8') as fp: ... print("""\ ... From: aperson@example.com ... To: ant@example.com ... Subject: testing ... Message-ID: ... ... This is a test message. ... """, file=fp) Create a mailing list to inject this message into. >>> mlist = create_list('ant@example.com') >>> transaction.commit() The mailing list's incoming queue is empty. >>> from mailman.testing.helpers import get_queue_messages >>> get_queue_messages('in') [] By default, messages are injected into the incoming queue. >>> command('mailman inject --filename ' + filename + ' ant@example.com') >>> items = get_queue_messages('in') >>> len(items) 1 >>> print(items[0].msg.as_string()) From: aperson@example.com To: ant@example.com Subject: testing Message-ID: ... Date: ... This is a test message. And the message is destined for ant@example.com. >>> dump_msgdata(items[0].msgdata) _parsemsg : False listid : ant.example.com original_size: 252 version : 3 But a different queue can be specified on the command line. :: >>> command('mailman inject --queue virgin --filename ' + ... filename + ' ant@example.com') >>> get_queue_messages('in') [] >>> items = get_queue_messages('virgin') >>> len(items) 1 >>> print(items[0].msg.as_string()) From: aperson@example.com To: ant@example.com Subject: testing Message-ID: ... Date: ... This is a test message. >>> dump_msgdata(items[0].msgdata) _parsemsg : False listid : ant.example.com original_size: 252 version : 3 Standard input ============== The message text can also be provided on standard input. :: >>> stdin = """\ ... From: bperson@example.com ... To: ant@example.com ... Subject: another test ... Message-ID: ... ... This is another test message. ... """ >>> command('mailman inject --filename - ant@example.com', input=stdin) >>> items = get_queue_messages('in') >>> len(items) 1 >>> print(items[0].msg.as_string()) From: bperson@example.com To: ant@example.com Subject: another test Message-ID: ... Date: ... This is another test message. >>> dump_msgdata(items[0].msgdata) _parsemsg : False listid : ant.example.com original_size: 260 version : 3 Metadata ======== Additional metadata keys can be provided on the command line. These key/value pairs get added to the message metadata dictionary when the message is injected. :: >>> command('mailman inject --filename ' + filename + ... ' -m foo=one -m bar=two ant@example.com') >>> items = get_queue_messages('in') >>> dump_msgdata(items[0].msgdata) _parsemsg : False bar : two foo : one listid : ant.example.com original_size: 252 version : 3 mailman-3.2.2/src/mailman/commands/docs/lists.rst0000644000175000017500000000656713421245331023114 0ustar maxkingmaxking00000000000000========================= Command line list display ========================= A system administrator can display all the mailing lists via the command line. When there are no mailing lists, a helpful message is displayed. :: >>> command = cli('mailman.commands.cli_lists.lists') >>> command('mailman lists') No matching mailing lists found When there are a few mailing lists, they are shown in alphabetical order by their fully qualified list names, with a description. :: >>> from mailman.interfaces.domain import IDomainManager >>> from zope.component import getUtility >>> getUtility(IDomainManager).add('example.net') >>> mlist_1 = create_list('list-one@example.com') >>> mlist_1.description = 'List One' >>> mlist_2 = create_list('list-two@example.com') >>> mlist_2.description = 'List Two' >>> mlist_3 = create_list('list-one@example.net') >>> mlist_3.description = 'List One in Example.Net' >>> command('mailman lists') 3 matching mailing lists found: list-one@example.com list-one@example.net list-two@example.com Names ===== You can display the mailing list names with their posting addresses, using the ``--names/-n`` switch. >>> command('mailman lists --names') 3 matching mailing lists found: list-one@example.com [List-one] list-one@example.net [List-one] list-two@example.com [List-two] Descriptions ============ You can also display the mailing list descriptions, using the ``--descriptions/-d`` option. >>> command('mailman lists --descriptions --names') 3 matching mailing lists found: list-one@example.com [List-one] - List One list-one@example.net [List-one] - List One in Example.Net list-two@example.com [List-two] - List Two Maybe you want the descriptions but not the names. >>> command('mailman lists --descriptions --no-names') 3 matching mailing lists found: list-one@example.com - List One list-one@example.net - List One in Example.Net list-two@example.com - List Two Less verbosity ============== There's also a ``--quiet/-q`` switch which reduces the verbosity a bit. >>> command('mailman lists --quiet') list-one@example.com list-one@example.net list-two@example.com Specific domain =============== You can narrow the search down to a specific domain with the --domain option. A helpful message is displayed if no matching domains are given. >>> command('mailman lists --domain example.org') No matching mailing lists found But if a matching domain is given, only mailing lists in that domain are shown. >>> command('mailman lists --domain example.net') 1 matching mailing lists found: list-one@example.net More than one ``--domain`` argument can be given; then all mailing lists in matching domains are shown. >>> command('mailman lists --domain example.com --domain example.net') 3 matching mailing lists found: list-one@example.com list-one@example.net list-two@example.com Advertised lists ================ Mailing lists can be "advertised" meaning their existence is public knowledge. Non-advertised lists are considered private. Display through the command line can select on this attribute. :: >>> mlist_1.advertised = False >>> command('mailman lists --advertised') 2 matching mailing lists found: list-one@example.net list-two@example.com mailman-3.2.2/src/mailman/commands/docs/members.rst0000644000175000017500000002035513421245331023377 0ustar maxkingmaxking00000000000000================ Managing members ================ The ``mailman members`` command allows a site administrator to display, add, and remove members from a mailing list. >>> command = cli('mailman.commands.cli_members.members') Listing members =============== You can list all the members of a mailing list by calling the command with no options. To start with, there are no members of the mailing list. >>> ant = create_list('ant@example.com') >>> command('mailman members ant.example.com') ant.example.com has no members Once the mailing list add some members, they will be displayed. :: >>> from mailman.testing.helpers import subscribe >>> subscribe(ant, 'Anne', email='anne@example.com') on ant@example.com as MemberRole.member> >>> subscribe(ant, 'Bart', email='bart@example.com') on ant@example.com as MemberRole.member> >>> command('mailman members ant.example.com') Anne Person Bart Person Members are displayed in alphabetical order based on their address. :: >>> subscribe(ant, 'Anne', email='anne@aaaxample.com') on ant@example.com as MemberRole.member> >>> command('mailman members ant.example.com') Anne Person Anne Person Bart Person You can also output this list to a file. :: >>> from tempfile import NamedTemporaryFile >>> filename = cleanups.enter_context(NamedTemporaryFile()).name >>> command('mailman members -o ' + filename + ' ant.example.com') >>> with open(filename, 'r', encoding='utf-8') as fp: ... print(fp.read()) Anne Person Anne Person Bart Person The output file can also be standard out. >>> command('mailman members -o - ant.example.com') Anne Person Anne Person Bart Person Filtering on delivery mode -------------------------- You can limit output to just the regular non-digest members... :: >>> member = ant.members.get_member('anne@example.com') >>> from mailman.interfaces.member import DeliveryMode >>> member.preferences.delivery_mode = DeliveryMode.plaintext_digests >>> command('mailman members --regular ant.example.com') Anne Person Bart Person ...or just the digest members. Furthermore, you can either display all digest members... :: >>> member = ant.members.get_member('anne@aaaxample.com') >>> member.preferences.delivery_mode = DeliveryMode.mime_digests >>> command('mailman members --digest any ant.example.com') Anne Person Anne Person ...just plain text digest members... >>> command('mailman members --digest plaintext ant.example.com') Anne Person ...or just MIME digest members. :: >>> command('mailman members --digest mime ant.example.com') Anne Person Filtering on delivery status ---------------------------- You can also filter the display on the member's delivery status. By default, all members are displayed, but you can filter out only those whose delivery status is enabled... :: >>> from mailman.interfaces.member import DeliveryStatus >>> member = ant.members.get_member('anne@aaaxample.com') >>> member.preferences.delivery_status = DeliveryStatus.by_moderator >>> member = ant.members.get_member('bart@example.com') >>> member.preferences.delivery_status = DeliveryStatus.by_user >>> member = subscribe(ant, 'Cris', email='cris@example.com') >>> member.preferences.delivery_status = DeliveryStatus.unknown >>> member = subscribe(ant, 'Dave', email='dave@example.com') >>> member.preferences.delivery_status = DeliveryStatus.enabled >>> member = subscribe(ant, 'Elle', email='elle@example.com') >>> member.preferences.delivery_status = DeliveryStatus.by_bounces >>> command('mailman members --nomail enabled ant.example.com') Anne Person Dave Person ...or disabled by the user... >>> command('mailman members --nomail byuser ant.example.com') Bart Person ...or disabled by the list administrator (or moderator)... >>> command('mailman members --nomail byadmin ant.example.com') Anne Person ...or by the bounce processor... >>> command('mailman members --nomail bybounces ant.example.com') Elle Person ...or for unknown (legacy) reasons. >>> command('mailman members --nomail unknown ant.example.com') Cris Person You can also display all members who have delivery disabled for any reason. :: >>> command('mailman members --nomail any ant.example.com') Anne Person Bart Person Cris Person Elle Person Adding members ============== You can add members to a mailing list from the command line. To do so, you need a file containing email addresses and full names that can be parsed by ``email.utils.parseaddr()``. :: >>> bee = create_list('bee@example.com') >>> with open(filename, 'w', encoding='utf-8') as fp: ... print("""\ ... aperson@example.com ... Bart Person ... cperson@example.com (Cate Person) ... """, file=fp) >>> command('mailman members --add ' + filename + ' bee.example.com') >>> from operator import attrgetter >>> dump_list(bee.members.addresses, key=attrgetter('email')) aperson@example.com Bart Person Cate Person You can also specify ``-`` as the filename, in which case the addresses are taken from standard input. :: >>> stdin = """\ ... dperson@example.com ... Elly Person ... fperson@example.com (Fred Person) ... """ >>> command('mailman members --add - bee.example.com', input=stdin) >>> dump_list(bee.members.addresses, key=attrgetter('email')) aperson@example.com Bart Person Cate Person dperson@example.com Elly Person Fred Person Blank lines and lines that begin with '#' are ignored. :: >>> with open(filename, 'w', encoding='utf-8') as fp: ... print("""\ ... gperson@example.com ... # hperson@example.com ... ... iperson@example.com ... """, file=fp) >>> command('mailman members --add ' + filename + ' bee.example.com') >>> dump_list(bee.members.addresses, key=attrgetter('email')) aperson@example.com Bart Person Cate Person dperson@example.com Elly Person Fred Person gperson@example.com iperson@example.com Addresses which are already subscribed are ignored, although a warning is printed. :: >>> with open(filename, 'w', encoding='utf-8') as fp: ... print("""\ ... gperson@example.com ... aperson@example.com ... jperson@example.com ... """, file=fp) >>> command('mailman members --add ' + filename + ' bee.example.com') Already subscribed (skipping): gperson@example.com Already subscribed (skipping): aperson@example.com >>> dump_list(bee.members.addresses, key=attrgetter('email')) aperson@example.com Bart Person Cate Person dperson@example.com Elly Person Fred Person gperson@example.com iperson@example.com jperson@example.com Displaying members ================== With no arguments, the command displays all members of the list. >>> command('mailman members bee.example.com') aperson@example.com Bart Person Cate Person dperson@example.com Elly Person Fred Person gperson@example.com iperson@example.com jperson@example.com mailman-3.2.2/src/mailman/commands/docs/membership.rst0000644000175000017500000002461713421245331024105 0ustar maxkingmaxking00000000000000============================ Membership changes via email ============================ Membership changes such as joining and leaving a mailing list, can be effected via the email interface. The Mailman email commands ``join``, ``leave``, and ``confirm`` are used. Joining a mailing list ====================== The mail command ``join`` subscribes an email address to the mailing list. ``subscribe`` is an alias for ``join``. >>> from mailman.commands.eml_membership import Join >>> from mailman.utilities.string import wrap >>> join = Join() >>> print(join.name) join >>> print(wrap(join.description)) You will be asked to confirm your subscription request and you may be issued a provisional password. By using the 'digest' option, you can specify whether you want digest delivery or not. If not specified, the mailing list's default delivery mode will be used. >>> print(join.argument_description) [digest=] No address to join ------------------ >>> mlist = create_list('alpha@example.com') >>> mlist.send_welcome_message = False When no address argument is given, the message's From address will be used. If that's missing though, then an error is returned. :: >>> from mailman.runners.command import Results >>> results = Results() >>> from mailman.email.message import Message >>> print(join.process(mlist, Message(), {}, (), results)) ContinueProcessing.no >>> print(results) The results of your email command are provided below. join: No valid address found to subscribe The ``subscribe`` command is an alias. >>> from mailman.commands.eml_membership import Subscribe >>> subscribe = Subscribe() >>> print(subscribe.name) subscribe >>> results = Results() >>> print(subscribe.process(mlist, Message(), {}, (), results)) ContinueProcessing.no >>> print(results) The results of your email command are provided below. subscribe: No valid address found to subscribe Joining the sender ------------------ When the message has a ``From`` field, that address will be subscribed. >>> msg = message_from_string("""\ ... From: Anne Person ... ... """) >>> results = Results() >>> print(join.process(mlist, msg, {}, (), results)) ContinueProcessing.yes >>> print(results) The results of your email command are provided below. Confirmation email sent to Anne Person Anne is not yet a member of the mailing list because she must confirm her subscription request first. >>> print(mlist.members.get_member('anne@example.com')) None Mailman has sent her the confirmation message. >>> from mailman.testing.helpers import get_queue_messages >>> items = get_queue_messages('virgin') >>> len(items) 1 >>> print(items[0].msg.as_string()) MIME-Version: 1.0 ... Subject: confirm ... From: alpha-confirm+...@example.com To: anne@example.com ... Email Address Registration Confirmation Hello, this is the GNU Mailman server at example.com. We have received a registration request for the email address anne@example.com Before you can start using GNU Mailman at this site, you must first confirm that this is your email address. You can do this by replying to this me... keeping the Subject header intact. If you do not wish to register this email address, simply disregard this message. If you think you are being maliciously subscribed to the list, or have any other questions, you may contact alpha-owner@example.com Anne confirms her registration. :: >>> def extract_token(message): ... return str(message['subject']).split()[1].strip() >>> token = extract_token(items[0].msg) >>> from mailman.commands.eml_confirm import Confirm >>> confirm = Confirm() >>> msg = message_from_string("""\ ... To: alpha-confirm+{token}@example.com ... From: anne@example.com ... Subject: Re: confirm {token} ... ... """.format(token=token)) >>> results = Results() >>> print(confirm.process(mlist, msg, {}, (token,), results)) ContinueProcessing.no >>> print(results) The results of your email command are provided below. Confirmed Anne is now a member of the mailing list. >>> mlist.members.get_member('anne@example.com') on alpha@example.com as MemberRole.member> Joining a second list --------------------- >>> mlist_2 = create_list('baker@example.com') >>> msg = message_from_string("""\ ... From: Anne Person ... ... """) >>> print(join.process(mlist_2, msg, {}, (), Results())) ContinueProcessing.yes Anne is not a member of the mailing list. >>> print(mlist_2.members.get_member('anne@example.com')) None One Anne confirms this subscription, she becomes a member of the mailing list. :: >>> items = get_queue_messages('virgin') >>> len(items) 1 >>> token = extract_token(items[0].msg) >>> msg = message_from_string("""\ ... To: baker-confirm+{token}@example.com ... From: anne@example.com ... Subject: Re: confirm {token} ... ... """.format(token=token)) >>> results = Results() >>> print(confirm.process(mlist_2, msg, {}, (token,), results)) ContinueProcessing.no >>> print(results) The results of your email command are provided below. Confirmed >>> print(mlist_2.members.get_member('anne@example.com')) on baker@example.com as MemberRole.member> Leaving a mailing list ====================== The mail command ``leave`` unsubscribes an email address from the mailing list. ``unsubscribe`` is an alias for ``leave``. >>> from mailman.commands.eml_membership import Leave >>> leave = Leave() >>> print(leave.name) leave >>> print(leave.description) Leave this mailing list. You may be asked to confirm your request. Anne is a member of the ``baker@example.com`` mailing list, when she decides to leave it. Because the mailing list allows for *open* unsubscriptions (i.e. no confirmation is needed), when she sends a message to the ``-leave`` address for the list, she is immediately removed. >>> from mailman.interfaces.mailinglist import SubscriptionPolicy >>> mlist_2.unsubscription_policy = SubscriptionPolicy.open >>> mlist.unsubscription_policy = SubscriptionPolicy.open >>> results = Results() >>> print(leave.process(mlist_2, msg, {}, (), results)) ContinueProcessing.yes >>> print(results) The results of your email command are provided below. Anne Person left baker@example.com Anne is no longer a member of the mailing list. >>> print(mlist_2.members.get_member('anne@example.com')) None Anne does not need to leave a mailing list with the same email address she's subscribe with. Any of her registered, linked, and validated email addresses will do. :: >>> from mailman.interfaces.usermanager import IUserManager >>> from zope.component import getUtility >>> anne = getUtility(IUserManager).get_user('anne@example.com') >>> address = anne.register('anne.person@example.org') >>> results = Results() >>> print(mlist.members.get_member('anne@example.com')) on alpha@example.com as MemberRole.member> >>> msg = message_from_string("""\ ... To: alpha-leave@example.com ... From: anne.person@example.org ... ... """) Since Anne's alternative address has not yet been verified, it can't be used to unsubscribe Anne from the alpha mailing list. :: >>> print(leave.process(mlist, msg, {}, (), results)) ContinueProcessing.no >>> print(results) The results of your email command are provided below. Invalid or unverified email address: anne.person@example.org >>> print(mlist.members.get_member('anne@example.com')) on alpha@example.com as MemberRole.member> Once Anne has verified her alternative address though, it can be used to unsubscribe her from the list. :: >>> from mailman.utilities.datetime import now >>> address.verified_on = now() >>> results = Results() >>> print(leave.process(mlist, msg, {}, (), results)) ContinueProcessing.yes >>> print(results) The results of your email command are provided below. Anne Person left alpha@example.com >>> print(mlist.members.get_member('anne@example.com')) None Confirmations ============= Bart wants to join the alpha list, so he sends his subscription request. :: >>> msg = message_from_string("""\ ... From: Bart Person ... ... """) >>> print(join.process(mlist, msg, {}, (), Results())) ContinueProcessing.yes There are two messages in the virgin queue, one of which is the confirmation message. >>> for item in get_queue_messages('virgin'): ... if str(item.msg['subject']).startswith('confirm'): ... break ... else: ... raise AssertionError('No confirmation message') >>> token = extract_token(item.msg) Bart replies to the original message, specifically keeping the Subject header intact except for any prefix. Mailman matches the token and confirms Bart as a user of the system. :: >>> msg = message_from_string("""\ ... From: Bart Person ... To: alpha-confirm+{token}@example.com ... Subject: Re: confirm {token} ... ... """.format(token=token)) >>> results = Results() >>> print(confirm.process(mlist, msg, {}, (token,), results)) ContinueProcessing.no >>> print(results) The results of your email command are provided below. Confirmed Now Bart is now a member of the mailing list. >>> print(mlist.members.get_member('bart@example.com')) on alpha@example.com as MemberRole.member> mailman-3.2.2/src/mailman/commands/docs/qfile.rst0000644000175000017500000000266113421245331023045 0ustar maxkingmaxking00000000000000=================== Dumping queue files =================== The ``qfile`` command dumps the contents of a queue pickle file. This is especially useful when you have shunt files you want to inspect. Pretty printing =============== By default, the ``qfile`` command pretty prints the contents of a queue pickle file to standard output. :: >>> command = cli('mailman.commands.cli_qfile.qfile') Let's say Mailman shunted a message file. :: >>> msg = message_from_string("""\ ... From: aperson@example.com ... To: ant@example.com ... Subject: Uh oh ... ... I borkeded Mailman. ... """) >>> shuntq = config.switchboards['shunt'] >>> basename = shuntq.enqueue(msg, foo=7, bar='baz', bad='yes') Once we've figured out the file name of the shunted message, we can print it. :: >>> from os.path import join >>> qfile = join(shuntq.queue_directory, basename + '.pck') >>> command('mailman qfile ' + qfile) [----- start pickle -----] <----- start object 1 -----> From: aperson@example.com To: ant@example.com Subject: Uh oh I borkeded Mailman. <----- start object 2 -----> {'_parsemsg': False, 'bad': 'yes', 'bar': 'baz', 'foo': 7, 'version': 3} [----- end pickle -----] Maybe we don't want to print the contents of the file though, in case we want to enter the interactive prompt. >>> command('mailman qfile --no-print ' + qfile) mailman-3.2.2/src/mailman/commands/docs/remove.rst0000644000175000017500000000151113421245331023233 0ustar maxkingmaxking00000000000000========================= Command line list removal ========================= A system administrator can remove mailing lists by the command line. :: >>> create_list('ant@example.com') >>> command = cli('mailman.commands.cli_lists.remove') >>> command('mailman remove ant@example.com') Removed list: ant@example.com >>> from mailman.interfaces.listmanager import IListManager >>> from zope.component import getUtility >>> list_manager = getUtility(IListManager) >>> print(list_manager.get('ant@example.com')) None You can also remove lists quietly. :: >>> create_list('ant@example.com') >>> command('mailman remove ant@example.com --quiet') >>> print(list_manager.get('ant@example.com')) None mailman-3.2.2/src/mailman/commands/docs/shell.rst0000644000175000017500000001114513421245331023051 0ustar maxkingmaxking00000000000000========================== Operating on mailing lists ========================== The ``shell`` (alias: ``withlist``) command is a pretty powerful way to operate on mailing lists from the command line. This command allows you to interact with a list at a Python prompt, or process one or more mailing lists through custom made Python functions. Getting detailed help ===================== Because ``shell`` is so complex, you might want to read the detailed help. :: >>> command = cli('mailman.commands.cli_withlist.shell') >>> command('mailman shell --details') This script provides you with a general framework for interacting with a mailing list. ... Running a function ================== By putting a Python function somewhere on your ``sys.path``, you can have ``shell`` call that function on a given mailing list. >>> import os, sys >>> old_path = sys.path[:] >>> sys.path.insert(0, config.VAR_DIR) .. cleanup >>> ignore = cleanups.callback(setattr, sys, 'path', old_path) The function takes at least a single argument, the mailing list. :: >>> with open(os.path.join(config.VAR_DIR, 'showme.py'), 'w') as fp: ... print("""\ ... def showme(mlist): ... print("The list's name is", mlist.fqdn_listname) ... ... def displayname(mlist): ... print("The list's display name is", mlist.display_name) ... ... def changeme(mlist, display_name): ... mlist.display_name = display_name ... """, file=fp) If the name of the function is the same as the module, then you only need to name the function once. >>> mlist = create_list('ant@example.com') >>> command('mailman shell -l ant@example.com --run showme') The list's name is ant@example.com The function's name can also be different than the modules name. In that case, just give the full module path name to the function you want to call. >>> command('mailman shell -l ant@example.com --run showme.displayname') The list's display name is Ant Passing arguments ================= Your function can also accept an arbitrary number of arguments. Every command line argument after the callable name is passed as a positional argument to the function. For example, to change the mailing list's display name, you can do this:: >>> command('mailman shell -l ant@example.com --run showme.changeme ANT!') >>> print(mlist.display_name) ANT! Multiple lists ============== You can run a command over more than one list by using a regular expression in the ``listname`` argument. To indicate a regular expression is used, the string must start with a caret. :: >>> mlist_2 = create_list('badger@example.com') >>> mlist_3 = create_list('badboys@example.com') >>> command('mailman shell --run showme.displayname -l ^.*example.com') The list's display name is ANT! The list's display name is Badboys The list's display name is Badger >>> command('mailman shell --run showme.displayname -l ^bad.*') The list's display name is Badboys The list's display name is Badger >>> command('mailman shell --run showme.displayname -l ^foo') Interactive use =============== You can also get an interactive prompt which allows you to inspect a live Mailman system directly. Through the ``mailman.cfg`` file, you can set the prompt and banner, and you can choose between the standard Python REPL_ or IPython. If the `GNU readline`_ library is available, it will be enabled automatically, giving you command line editing and other features. You can also set the ``[shell]history_file`` variable in the ``mailman.cfg`` file and when the normal Python REPL is used, your interactive commands will be written to and read from this file. Note that the ``$PYTHONSTARTUP`` environment variable will also be honored if set, and any file named by this variable will be read at start up time. It's common practice to *also* enable GNU readline history in a ``$PYTHONSTARTUP`` file and if you do this, be aware that it will interact badly with ``[shell]history_file``, causing your history to be written twice. To disable this when using the interactive ``shell`` command, do something like:: $ PYTHONSTARTUP= mailman shell to temporarily unset the environment variable. IPython ------- You can use IPython_ as the interactive shell by setting the ``[shell]use_ipython`` variables in your `mailman.cfg` file to ``yes``. IPython must be installed and available on your system When using IPython, the ``[shell]history_file`` is not used. .. _IPython: http://ipython.org/ .. _REPL: https://en.wikipedia.org/wiki/REPL .. _`GNU readline`: https://docs.python.org/3/library/readline.html mailman-3.2.2/src/mailman/commands/docs/status.rst0000644000175000017500000000167613441266421023302 0ustar maxkingmaxking00000000000000============== Getting status ============== The status of the Mailman master process can be queried from the command line. It's clear at this point that nothing is running. :: >>> command = cli('mailman.commands.cli_status.status') The status is printed to stdout and a status code is returned. >>> command('mailman status') GNU Mailman is not running We can simulate the master starting up by acquiring its lock. >>> from datetime import timedelta >>> from flufl.lock import Lock >>> lock = Lock(config.LOCK_FILE) >>> lock.lock(timeout=timedelta(seconds=20)) >>> ignore = cleanups.callback(lock.unlock, unconditionally=True) Getting the status confirms that the master is running. >>> command('mailman status') GNU Mailman is running (master pid: ... We shut down the master and confirm the status. >>> lock.unlock(unconditionally=True) >>> command('mailman status') GNU Mailman is not running mailman-3.2.2/src/mailman/commands/docs/unshunt.rst0000644000175000017500000000670113421245331023450 0ustar maxkingmaxking00000000000000======= Unshunt ======= When errors occur while processing email messages, the messages will end up in the ``shunt`` queue. The ``unshunt`` command allows system administrators to manage the shunt queue. :: >>> command = cli('mailman.commands.cli_unshunt.unshunt') Let's say there is a message in the shunt queue. :: >>> msg = message_from_string("""\ ... From: aperson@example.com ... To: test@example.com ... Subject: A broken message ... Message-ID: ... ... """) >>> shuntq = config.switchboards['shunt'] >>> len(list(shuntq.files)) 0 >>> base_name = shuntq.enqueue(msg, {}) >>> len(list(shuntq.files)) 1 The ``unshunt`` command by default moves the message back to the incoming queue. :: >>> inq = config.switchboards['in'] >>> len(list(inq.files)) 0 >>> command('mailman unshunt') >>> from mailman.testing.helpers import get_queue_messages >>> items = get_queue_messages('in') >>> len(items) 1 >>> print(items[0].msg.as_string()) From: aperson@example.com To: test@example.com Subject: A broken message Message-ID: ``unshunt`` moves all shunt queue messages. :: >>> msg = message_from_string("""\ ... From: aperson@example.com ... To: test@example.com ... Subject: A broken message ... Message-ID: ... ... """) >>> base_name = shuntq.enqueue(msg, {}) >>> msg = message_from_string("""\ ... From: aperson@example.com ... To: test@example.com ... Subject: A broken message ... Message-ID: ... ... """) >>> base_name = shuntq.enqueue(msg, {}) >>> len(list(shuntq.files)) 2 >>> command('mailman unshunt') >>> items = get_queue_messages('in') >>> len(items) 2 >>> sorted(item.msg['message-id'] for item in items) ['', ''] Return to the original queue ============================ While the messages in the shunt queue are generally returned to the incoming queue, if the error occurred while the message was being processed from a different queue, it will be returned to the queue it came from. >>> msg = message_from_string("""\ ... From: aperson@example.com ... To: test@example.com ... Subject: A broken message ... Message-ID: ... ... """) The queue that the message comes from is in message metadata. :: >>> base_name = shuntq.enqueue(msg, {}, whichq='bounces') >>> len(list(shuntq.files)) 1 >>> len(list(config.switchboards['bounces'].files)) 0 The message is automatically re-queued to the bounces queue. :: >>> command('mailman unshunt') >>> len(list(shuntq.files)) 0 >>> items = get_queue_messages('bounces') >>> len(items) 1 >>> print(items[0].msg.as_string()) From: aperson@example.com To: test@example.com Subject: A broken message Message-ID: Discarding all shunted messages =============================== If you don't care about the shunted messages, just discard them. :: >>> msg = message_from_string("""\ ... From: aperson@example.com ... To: test@example.com ... Subject: A broken message ... Message-ID: ... ... """) >>> base_name = shuntq.enqueue(msg, {}) >>> command('mailman unshunt --discard') The messages are now gone. >>> items = get_queue_messages('in') >>> len(items) 0 mailman-3.2.2/src/mailman/commands/docs/version.rst0000644000175000017500000000040513421245331023424 0ustar maxkingmaxking00000000000000==================== Printing the version ==================== You can print the Mailman version number by invoking the ``version`` command. >>> command = cli('mailman.commands.cli_version.version') >>> command('mailman version') GNU Mailman 3... mailman-3.2.2/src/mailman/commands/eml_confirm.py0000644000175000017500000000633313442110351023123 0ustar maxkingmaxking00000000000000# Copyright (C) 2009-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """The 'confirm' email command.""" from mailman.core.i18n import _ from mailman.interfaces.command import ContinueProcessing, IEmailCommand from mailman.interfaces.subscriptions import ISubscriptionManager, TokenOwner from public import public from zope.interface import implementer @public @implementer(IEmailCommand) class Confirm: """The email 'confirm' command.""" name = 'confirm' argument_description = 'token' description = _('Confirm a subscription request.') short_description = description def process(self, mlist, msg, msgdata, arguments, results): """See `IEmailCommand`.""" # The token must be in the arguments. if len(arguments) == 0: print(_('No confirmation token found'), file=results) return ContinueProcessing.no # Make sure we don't try to confirm the same token more than once. token = arguments[0] tokens = getattr(results, 'confirms', set()) if token in tokens: # Do not try to confirm this one again. return ContinueProcessing.no tokens.add(token) results.confirms = tokens try: new_token, token_owner, member = ISubscriptionManager( mlist).confirm(token) if new_token is None: assert token_owner is TokenOwner.no_one, token_owner # We can't assert anything about member. It will be None when # the workflow we're confirming is an unsubscription request, # and non-None when we're confirming a subscription request. # This class doesn't know which is happening. succeeded = True elif token_owner is TokenOwner.moderator: # This must have been a confirm-then-moderator subscription. assert new_token != token assert member is None, member succeeded = True else: assert token_owner is not TokenOwner.no_one, token_owner assert member is None, member succeeded = False except LookupError: # The token must not exist in the database. succeeded = False if succeeded: print(_('Confirmed'), file=results) # After the 'confirm' command, do not process any other commands in # the email. return ContinueProcessing.no print(_('Confirmation token did not match'), file=results) return ContinueProcessing.no mailman-3.2.2/src/mailman/commands/eml_echo.py0000644000175000017500000000254113442110351022401 0ustar maxkingmaxking00000000000000# Copyright (C) 2002-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """The email command 'echo'.""" from mailman.core.i18n import _ from mailman.interfaces.command import ContinueProcessing, IEmailCommand from public import public from zope.interface import implementer SPACE = ' ' @public @implementer(IEmailCommand) class Echo: """The email 'echo' command.""" name = 'echo' argument_description = '[args]' description = _('Echo back your arguments.') short_description = description def process(self, mlist, msg, msgdata, arguments, results): """See `IEmailCommand`.""" print('echo', SPACE.join(arguments), file=results) return ContinueProcessing.yes mailman-3.2.2/src/mailman/commands/eml_end.py0000644000175000017500000000275713442110351022242 0ustar maxkingmaxking00000000000000# Copyright (C) 2002-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """The email commands 'end' and 'stop'.""" from mailman.core.i18n import _ from mailman.interfaces.command import ContinueProcessing, IEmailCommand from public import public from zope.interface import implementer @public @implementer(IEmailCommand) class End: """The email 'end' command.""" name = 'end' argument_description = '' description = _('Stop processing commands.') short_description = description def process(self, mlist, msg, msgdata, arguments, results): """See `IEmailCommand`.""" # Ignore all arguments. return ContinueProcessing.no @public class Stop(End): """The email 'stop' command (an alias for 'end').""" name = 'stop' description = _("An alias for 'end'.") short_description = description mailman-3.2.2/src/mailman/commands/eml_help.py0000644000175000017500000000556513442110351022424 0ustar maxkingmaxking00000000000000# Copyright (C) 2012-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """The email command 'help'.""" from mailman.config import config from mailman.core.i18n import _ from mailman.interfaces.command import ContinueProcessing, IEmailCommand from mailman.utilities.string import wrap from public import public from zope.interface import implementer SPACE = ' ' @public @implementer(IEmailCommand) class Help: """The email 'help' command.""" name = 'help' argument_description = '[command]' description = _('Get help about available email commands.') short_description = description def process(self, mlist, msg, msgdata, arguments, results): """See `IEmailCommand`.""" # With no argument, print the command and a short description, which # is contained in the short_description attribute. if len(arguments) == 0: length = max(len(command) for command in config.commands) format = '{{0: <{0}s}} - {{1}}'.format(length) for command_name in sorted(config.commands): command = config.commands[command_name] short_description = getattr( command, 'short_description', _('n/a')) print(format.format(command.name, short_description), file=results) return ContinueProcessing.yes elif len(arguments) == 1: command_name = arguments[0] command = config.commands.get(command_name) if command is None: print(_('$self.name: no such command: $command_name'), file=results) return ContinueProcessing.no print('{} {}'.format(command.name, command.argument_description), file=results) print(command.short_description, file=results) if command.short_description != command.description: print(wrap(command.description), file=results) return ContinueProcessing.yes else: printable_arguments = SPACE.join(arguments) # noqa: F841 print(_('$self.name: too many arguments: $printable_arguments'), file=results) return ContinueProcessing.no mailman-3.2.2/src/mailman/commands/eml_membership.py0000644000175000017500000002104213442110351023613 0ustar maxkingmaxking00000000000000# Copyright (C) 2002-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """The email commands 'join' and 'subscribe'.""" from email.utils import formataddr, parseaddr from mailman.core.i18n import _ from mailman.interfaces.command import ContinueProcessing, IEmailCommand from mailman.interfaces.member import DeliveryMode, MemberRole from mailman.interfaces.subscriptions import ( ISubscriptionManager, ISubscriptionService) from mailman.interfaces.usermanager import IUserManager from public import public from zope.component import getUtility from zope.interface import implementer def match_subscriber(email, display_name): # Return something matching the email which should be used as the # subscriber by the ISubscriptionManager interface. manager = getUtility(IUserManager) # Is there a user with a preferred address matching the email? user = manager.get_user(email) if user is not None: preferred = user.preferred_address if preferred is not None and preferred.email == email.lower(): return user # Is there an address matching the email? address = manager.get_address(email) if address is not None: return address # Make a new user and subscribe their first (and only) address. We can't # make the first address their preferred address because it hasn't been # verified yet. user = manager.make_user(email, display_name) return list(user.addresses)[0] @public @implementer(IEmailCommand) class Join: """The email 'join' command.""" name = 'join' # XXX 2012-02-29 BAW: DeliveryMode.summary is not yet supported. argument_description = '[digest=]' description = _("""\ You will be asked to confirm your subscription request and you may be issued a provisional password. By using the 'digest' option, you can specify whether you want digest delivery or not. If not specified, the mailing list's default delivery mode will be used. """) short_description = _('Join this mailing list.') def process(self, mlist, msg, msgdata, arguments, results): """See `IEmailCommand`.""" # Parse the arguments. delivery_mode = self._parse_arguments(arguments, results) if delivery_mode is ContinueProcessing.no: return ContinueProcessing.no display_name, email = parseaddr(msg['from']) # Address could be None or the empty string. if not email: email = msg.sender if not email: print(_('$self.name: No valid address found to subscribe'), file=results) return ContinueProcessing.no if isinstance(email, bytes): email = email.decode('ascii') # Have we already seen one join request from this user during the # processing of this email? joins = getattr(results, 'joins', set()) if email in joins: # Do not register this join. return ContinueProcessing.yes joins.add(email) results.joins = joins person = formataddr((display_name, email)) # noqa: F841 # Is this person already a member of the list? Search for all # matching memberships. members = getUtility(ISubscriptionService).find_members( email, mlist.list_id, MemberRole.member) if len(members) > 0: print(_('$person is already a member'), file=results) return ContinueProcessing.yes subscriber = match_subscriber(email, display_name) ISubscriptionManager(mlist).register(subscriber) print(_('Confirmation email sent to $person'), file=results) return ContinueProcessing.yes def _parse_arguments(self, arguments, results): """Parse command arguments. :param arguments: The sequences of arguments as given to the `process()` method. :param results: The results object. :return: The delivery mode, None, or ContinueProcessing.no on error. """ mode = DeliveryMode.regular for argument in arguments: parts = argument.split('=', 1) if len(parts) != 2 or parts[0] != 'digest': print(self.name, _('bad argument: $argument'), file=results) return ContinueProcessing.no mode = { 'no': DeliveryMode.regular, 'plain': DeliveryMode.plaintext_digests, 'mime': DeliveryMode.mime_digests, }.get(parts[1]) if mode is None: print(self.name, _('bad argument: $argument'), file=results) return ContinueProcessing.no return mode @public class Subscribe(Join): """The email 'subscribe' command (an alias for 'join').""" name = 'subscribe' description = _("An alias for 'join'.") short_description = description @public @implementer(IEmailCommand) class Leave: """The email 'leave' command.""" name = 'leave' argument_description = '' description = _("""Leave this mailing list. You may be asked to confirm your request.""") short_description = _('Leave this mailing list.') def process(self, mlist, msg, msgdata, arguments, results): """See `IEmailCommand`.""" email = msg.sender if not email: print(_('$self.name: No valid email address found to unsubscribe'), file=results) return ContinueProcessing.no user_manager = getUtility(IUserManager) user = user_manager.get_user(email) if user is None: print(_('No registered user for email address: $email'), file=results) return ContinueProcessing.no # The address that the -leave command was sent from, must be verified. # Otherwise you could link a bogus address to anyone's account, and # then send a leave command from that address. if user_manager.get_address(email).verified_on is None: print(_('Invalid or unverified email address: $email'), file=results) return ContinueProcessing.no already_left = msgdata.setdefault('leaves', set()) for user_address in user.addresses: # Only recognize verified addresses. if user_address.verified_on is None: continue member = mlist.members.get_member(user_address.email) if member is not None: break else: # There are two possible situations. Either none of the user's # addresses are subscribed to this mailing list, or this command # email *already* unsubscribed the user from the mailing list. # E.g. if a message was sent to the -leave address and it # contained the 'leave' command. Don't send a bogus response in # this case, just ignore subsequent leaves of the same address. if email not in already_left: print(_('$self.name: $email is not a member of ' '$mlist.fqdn_listname'), file=results) return ContinueProcessing.no if email in already_left: return ContinueProcessing.yes # Ignore any subsequent 'leave' commands. already_left.add(email) manager = ISubscriptionManager(mlist) token, token_owner, member = manager.unregister(user_address) person = formataddr((user.display_name, email)) # noqa: F841 if member is None: print(_('$person left $mlist.fqdn_listname'), file=results) else: print(_('Confirmation email sent to $person to leave' ' $mlist.fqdn_listname'), file=results) return ContinueProcessing.yes @public class Unsubscribe(Leave): """The email 'unsubscribe' command (an alias for 'leave').""" name = 'unsubscribe' description = _("An alias for 'leave'.") short_description = description mailman-3.2.2/src/mailman/commands/tests/0000755000175000017500000000000013445614541021431 5ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/commands/tests/__init__.py0000644000175000017500000000000013244427337023532 0ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/commands/tests/data/0000755000175000017500000000000013445614541022342 5ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/commands/tests/data/__init__.py0000644000175000017500000000000013421245331024430 0ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/commands/tests/data/no-runners.cfg0000644000175000017500000000203113442110351025110 0ustar maxkingmaxking00000000000000# Copyright (C) 2008-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . # Disable all runners. [runner.archive] start: no [runner.bounces] start: no [runner.command] start: no [runner.in] start: no [runner.lmtp] start: no [runner.nntp] start: no [runner.out] start: no [runner.pipeline] start: no [runner.retry] start: no [runner.shunt] start: no [runner.virgin] start: no mailman-3.2.2/src/mailman/commands/tests/test_cli_conf.py0000644000175000017500000000560013442110351024602 0ustar maxkingmaxking00000000000000# Copyright (C) 2013-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Test the conf subcommand.""" import unittest from click.testing import CliRunner from mailman.commands.cli_conf import conf from mailman.testing.layers import ConfigLayer from tempfile import NamedTemporaryFile class TestConf(unittest.TestCase): """Test the conf subcommand.""" layer = ConfigLayer def setUp(self): self._command = CliRunner() def test_cannot_access_nonexistent_section(self): result = self._command.invoke(conf, ('-s', 'thissectiondoesnotexist')) self.assertEqual(result.exit_code, 2) self.assertEqual( result.output, 'Usage: conf [OPTIONS]\n' 'Try "conf --help" for help.\n\n' 'Error: No such section: thissectiondoesnotexist\n') def test_cannot_access_nonexistent_section_and_key(self): result = self._command.invoke( conf, ('-s', 'thissectiondoesnotexist', '-k', 'nosuchkey')) self.assertEqual(result.exit_code, 2) self.assertEqual( result.output, 'Usage: conf [OPTIONS]\n' 'Try "conf --help" for help.\n\n' 'Error: No such section: thissectiondoesnotexist\n') def test_cannot_access_nonexistent_key(self): result = self._command.invoke( conf, ('-s', 'mailman', '-k', 'thiskeydoesnotexist')) self.assertEqual(result.exit_code, 2) self.assertEqual( result.output, 'Usage: conf [OPTIONS]\n' 'Try "conf --help" for help.\n\n' 'Error: Section mailman: No such key: thiskeydoesnotexist\n') def test_output_to_explicit_stdout(self): result = self._command.invoke( conf, ('-o', '-', '-s', 'shell', '-k', 'use_ipython')) self.assertEqual(result.exit_code, 0) self.assertEqual(result.output, 'no\n') def test_output_to_file(self): with NamedTemporaryFile() as outfp: result = self._command.invoke( conf, ('-o', outfp.name, '-s', 'shell', '-k', 'use_ipython')) self.assertEqual(result.exit_code, 0) with open(outfp.name, 'r', encoding='utf-8') as infp: self.assertEqual(infp.read(), 'no\n') mailman-3.2.2/src/mailman/commands/tests/test_cli_control.py0000644000175000017500000003162513442110351025343 0ustar maxkingmaxking00000000000000# Copyright (C) 2011-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Test some additional corner cases for starting/stopping.""" import os import sys import time import shutil import signal import socket import unittest from click.testing import CliRunner from contextlib import ExitStack, suppress from datetime import datetime, timedelta from flufl.lock import SEP from importlib_resources import path from mailman.bin.master import WatcherState from mailman.commands.cli_control import reopen, restart, start from mailman.config import config from mailman.testing.helpers import configuration from mailman.testing.layers import ConfigLayer from public import public from tempfile import TemporaryDirectory from unittest.mock import patch # For ../docs/control.rst @public def make_config(resources): cfg_path = resources.enter_context( path('mailman.commands.tests.data', 'no-runners.cfg')) # We have to patch the global config's filename attribute. The problem # here is that click does not support setting the -C option on the # parent command (i.e. `master`). # https://github.com/pallets/click/issues/831 resources.enter_context(patch.object(config, 'filename', str(cfg_path))) # For ../docs/control.rst @public def find_master(): # See if the master process is still running. until = timedelta(seconds=10) + datetime.now() while datetime.now() < until: time.sleep(0.1) with suppress(FileNotFoundError, ValueError, ProcessLookupError): with open(config.PID_FILE) as fp: pid = int(fp.read().strip()) os.kill(pid, 0) return pid return None @public def claim_lock(): # Fake an acquisition of the master lock by another process, which # subsequently goes stale. Start by finding a free process id. Yes, # this could race, but given that we're starting with our own PID and # searching downward, it's less likely. fake_pid = os.getpid() - 1 while fake_pid > 1: try: os.kill(fake_pid, 0) except ProcessLookupError: break fake_pid -= 1 else: raise RuntimeError('Cannot find free PID') # Lock acquisition logic taken from flufl.lock. claim_file = SEP.join(( config.LOCK_FILE, socket.getfqdn(), str(fake_pid), '0')) with open(config.LOCK_FILE, 'w') as fp: fp.write(claim_file) os.link(config.LOCK_FILE, claim_file) expiration_date = datetime.now() - timedelta(minutes=5) t = time.mktime(expiration_date.timetuple()) os.utime(claim_file, (t, t)) return claim_file @public def kill_with_extreme_prejudice(pid_or_pidfile=None): # 2016-12-03 barry: We have intermittent hangs during both local and CI # test suite runs where killing a runner or master process doesn't # terminate the process. In those cases, wait()ing on the child can # suspend the test process indefinitely. Locally, you have to C-c the # test process, but that still doesn't kill it; the process continues to # run in the background. If you then search for the process's pid and # SIGTERM it, it will usually exit, which is why I don't understand why # the above SIGTERM doesn't kill it sometimes. However, when run under # CI, the test suite will just hang until the CI runner times it out. It # would be better to figure out the underlying cause, because we have # definitely seen other situations where a runner process won't exit, but # for testing purposes we're just trying to clean up some resources so # after a brief attempt at SIGTERMing it, let's SIGKILL it and warn. if isinstance(pid_or_pidfile, str): try: with open(pid_or_pidfile, 'r') as fp: pid = int(fp.read()) except FileNotFoundError: # There's nothing to kill. return else: pid = pid_or_pidfile if pid is not None: os.kill(pid, signal.SIGTERM) until = timedelta(seconds=10) + datetime.now() while datetime.now() < until: try: if pid is None: os.wait3(os.WNOHANG) else: os.waitpid(pid, os.WNOHANG) except ChildProcessError: # This basically means we went one too many times around the # loop. The previous iteration successfully reaped the child. # Because the return status of wait3() and waitpid() are different # in those cases, it's easier just to catch the exception for # either call and exit. return time.sleep(0.1) else: if pid is None: # There's really not much more we can do because we have no pid to # SIGKILL. Just report the problem and continue. print('WARNING: NO CHANGE IN CHILD PROCESS STATES', file=sys.stderr) return print('WARNING: SIGTERM DID NOT EXIT PROCESS; SIGKILLing', file=sys.stderr) if pid is not None: os.kill(pid, signal.SIGKILL) until = timedelta(seconds=10) + datetime.now() while datetime.now() < until: try: os.waitpid(pid, os.WNOHANG) except ChildProcessError: # 2016-03-10 maxking: We are seeing ChildProcessError very # often in CI due to the os.waitpid on L155 above. This is # raised when there is no child process left. We are clearly in # the arena of a race condition where the process was killed # somewhere after we checked and before we tried to wait on # it. TOCTTOU problem. return time.sleep(0.1) else: print('WARNING: SIGKILL DID NOT EXIT PROCESS!', file=sys.stderr) @public def clean_stale_locks(): """Cleanup the master.pid and master.lck file, if they exist.""" # If the master process was force-killed during the test suite run, it is # possible that the stale pid file was left. Clean that file up. if os.path.exists(config.PID_FILE): os.unlink(config.PID_FILE) if os.path.exists(config.LOCK_FILE): os.unlink(config.LOCK_FILE) class TestControl(unittest.TestCase): layer = ConfigLayer maxDiff = None def setUp(self): self._command = CliRunner() self._tmpdir = TemporaryDirectory() self.addCleanup(self._tmpdir.cleanup) # Specify where to put the pid file; and make sure that the master # gets killed regardless of whether it gets started or not. self._pid_file = os.path.join(self._tmpdir.name, 'master-test.pid') self.addCleanup(kill_with_extreme_prejudice, self._pid_file) # Patch cli_control so that 1) it doesn't actually do a fork, since # that makes it impossible to avoid race conditions in the test; 2) # doesn't actually os.execl(). with ExitStack() as resources: resources.enter_context(patch( 'mailman.commands.cli_control.os.fork', # Pretend to be the child. return_value=0 )) self._execl = resources.enter_context(patch( 'mailman.commands.cli_control.os.execl')) resources.enter_context(patch( 'mailman.commands.cli_control.os.setsid')) resources.enter_context(patch( 'mailman.commands.cli_control.os.chdir')) resources.enter_context(patch( 'mailman.commands.cli_control.os.environ', os.environ.copy())) # Arrange for the mocks to be reverted when the test is over. self.addCleanup(resources.pop_all().close) def test_master_is_elsewhere_and_missing(self): with ExitStack() as resources: bin_dir = resources.enter_context(TemporaryDirectory()) old_master = os.path.join(config.BIN_DIR, 'master') new_master = os.path.join(bin_dir, 'master') shutil.move(old_master, new_master) resources.callback(shutil.move, new_master, old_master) results = self._command.invoke(start) # Argument #2 to the execl() call should be the path to the master # program, and the path should not exist. self.assertEqual( len(self._execl.call_args_list), 1, results.output) posargs, kws = self._execl.call_args_list[0] master_path = posargs[2] self.assertEqual(os.path.basename(master_path), 'master') self.assertFalse(os.path.exists(master_path), master_path) def test_master_is_elsewhere_and_findable(self): with ExitStack() as resources: bin_dir = resources.enter_context(TemporaryDirectory()) old_master = os.path.join(config.BIN_DIR, 'master') new_master = os.path.join(bin_dir, 'master') shutil.move(old_master, new_master) resources.callback(shutil.move, new_master, old_master) with configuration('paths.testing', bin_dir=bin_dir): results = self._command.invoke(start) # Argument #2 to the execl() call should be the path to the master # program, and the path should exist. self.assertEqual( len(self._execl.call_args_list), 1, results.output) posargs, kws = self._execl.call_args_list[0] master_path = posargs[2] self.assertEqual(os.path.basename(master_path), 'master') self.assertTrue(os.path.exists(master_path), master_path) def test_stale_lock_no_force(self): claim_file = claim_lock() self.addCleanup(os.remove, claim_file) self.addCleanup(os.remove, config.LOCK_FILE) result = self._command.invoke(start) self.assertEqual(result.exit_code, 2) self.assertEqual( result.output, 'Usage: start [OPTIONS]\n' 'Try "start --help" for help.\n\n' 'Error: A previous run of GNU Mailman did not exit cleanly ' '(stale_lock). Try using --force\n') def test_stale_lock_force(self): claim_file = claim_lock() self.addCleanup(os.remove, claim_file) self.addCleanup(os.remove, config.LOCK_FILE) # Don't test the results of this command. Because we're mocking # os.execl(), we'll end up raising the RuntimeError at the end of the # start() method, child branch. self._command.invoke(start, ('--force',)) self.assertEqual(len(self._execl.call_args_list), 1) posargs, kws = self._execl.call_args_list[0] self.assertIn('--force', posargs) class TestControlSimple(unittest.TestCase): layer = ConfigLayer maxDiff = None def setUp(self): self._command = CliRunner() def test_watcher_state_conflict(self): with patch('mailman.commands.cli_control.master_state', return_value=(WatcherState.conflict, object())): results = self._command.invoke(start) self.assertEqual(results.exit_code, 2) self.assertEqual( results.output, 'Usage: start [OPTIONS]\n' 'Try "start --help" for help.\n\n' 'Error: GNU Mailman is already running\n') def test_reopen(self): with patch('mailman.commands.cli_control.kill_watcher') as mock: result = self._command.invoke(reopen) mock.assert_called_once_with(signal.SIGHUP) self.assertEqual(result.output, 'Reopening the Mailman runners\n') def test_reopen_quiet(self): with patch('mailman.commands.cli_control.kill_watcher') as mock: result = self._command.invoke(reopen, ('--quiet',)) mock.assert_called_once_with(signal.SIGHUP) self.assertEqual(result.output, '') def test_restart(self): with patch('mailman.commands.cli_control.kill_watcher') as mock: result = self._command.invoke(restart) mock.assert_called_once_with(signal.SIGUSR1) self.assertEqual(result.output, 'Restarting the Mailman runners\n') def test_restart_quiet(self): with patch('mailman.commands.cli_control.kill_watcher') as mock: result = self._command.invoke(restart, ('--quiet',)) mock.assert_called_once_with(signal.SIGUSR1) self.assertEqual(result.output, '') mailman-3.2.2/src/mailman/commands/tests/test_cli_create.py0000644000175000017500000001130313442110351025115 0ustar maxkingmaxking00000000000000# Copyright (C) 2011-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Test the `mailman create` subcommand.""" import unittest from click.testing import CliRunner from mailman.app.lifecycle import create_list from mailman.commands.cli_lists import create, remove from mailman.interfaces.domain import IDomainManager from mailman.testing.layers import ConfigLayer from zope.component import getUtility class TestCreate(unittest.TestCase): layer = ConfigLayer def setUp(self): self._command = CliRunner() def test_cannot_create_duplicate_list(self): # Cannot create a mailing list if it already exists. create_list('ant@example.com') result = self._command.invoke(create, ('ant@example.com',)) self.assertEqual(result.exit_code, 2) self.assertEqual( result.output, 'Usage: create [OPTIONS] LISTNAME\n' 'Try "create --help" for help.\n\n' 'Error: List already exists: ant@example.com\n') def test_invalid_posting_address(self): # Cannot create a mailing list with an invalid posting address. result = self._command.invoke(create, ('foo',)) self.assertEqual(result.exit_code, 2) self.assertEqual( result.output, 'Usage: create [OPTIONS] LISTNAME\n' 'Try "create --help" for help.\n\n' 'Error: Illegal list name: foo\n') def test_invalid_owner_addresses(self): # Cannot create a list with invalid owner addresses. LP: #778687 result = self._command.invoke( create, ('-o', 'invalid', 'ant@example.com')) self.assertEqual(result.exit_code, 2) self.assertEqual( result.output, 'Usage: create [OPTIONS] LISTNAME\n' 'Try "create --help" for help.\n\n' 'Error: Illegal owner addresses: invalid\n') def test_create_without_domain_option(self): # The domain will be created if no domain options are specified. Use # the example.org domain since example.com is created by the test # suite so it would always already exist. result = self._command.invoke(create, ('ant@example.org',)) self.assertEqual(result.exit_code, 0) domain = getUtility(IDomainManager)['example.org'] self.assertEqual(domain.mail_host, 'example.org') def test_create_with_d(self): result = self._command.invoke(create, ('ant@example.org', '-d')) self.assertEqual(result.exit_code, 0) domain = getUtility(IDomainManager)['example.org'] self.assertEqual(domain.mail_host, 'example.org') def test_create_with_domain(self): result = self._command.invoke(create, ('ant@example.org', '--domain')) self.assertEqual(result.exit_code, 0) domain = getUtility(IDomainManager)['example.org'] self.assertEqual(domain.mail_host, 'example.org') def test_create_with_D(self): result = self._command.invoke(create, ('ant@example.org', '-D')) self.assertEqual(result.exit_code, 2) self.assertEqual( result.output, 'Usage: create [OPTIONS] LISTNAME\n' 'Try "create --help" for help.\n\n' 'Error: Undefined domain: example.org\n') def test_create_with_nodomain(self): result = self._command.invoke( create, ('ant@example.org', '--no-domain')) self.assertEqual(result.exit_code, 2) self.assertEqual( result.output, 'Usage: create [OPTIONS] LISTNAME\n' 'Try "create --help" for help.\n\n' 'Error: Undefined domain: example.org\n') class TestRemove(unittest.TestCase): layer = ConfigLayer def setUp(self): self._command = CliRunner() def test_remove_not_quiet_no_such_list(self): results = self._command.invoke(remove, ('ant@example.com',)) # It's not an error to try to remove a nonexistent list. self.assertEqual(results.exit_code, 0) self.assertEqual( results.output, 'No such list matching spec: ant@example.com\n') mailman-3.2.2/src/mailman/commands/tests/test_cli_digests.py0000644000175000017500000005157113442110351025327 0ustar maxkingmaxking00000000000000# Copyright (C) 2015-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Test the send-digests subcommand.""" import os import unittest from click.testing import CliRunner from datetime import timedelta from mailman.app.lifecycle import create_list from mailman.commands.cli_digests import digests from mailman.config import config from mailman.interfaces.digests import DigestFrequency from mailman.interfaces.member import DeliveryMode from mailman.runners.digest import DigestRunner from mailman.testing.helpers import ( get_queue_messages, make_testable_runner, specialized_message_from_string as mfs, subscribe) from mailman.testing.layers import ConfigLayer from mailman.utilities.datetime import now as right_now class TestSendDigests(unittest.TestCase): layer = ConfigLayer def setUp(self): self._mlist = create_list('ant@example.com') self._mlist.digests_enabled = True self._mlist.digest_size_threshold = 100000 self._mlist.send_welcome_message = False self._command = CliRunner() self._handler = config.handlers['to-digest'] self._runner = make_testable_runner(DigestRunner, 'digest') # The mailing list needs at least one digest recipient. member = subscribe(self._mlist, 'Anne') member.preferences.delivery_mode = DeliveryMode.plaintext_digests def test_send_one_digest_by_list_id(self): msg = mfs("""\ To: ant@example.com From: anne@example.com Subject: message 1 """) self._handler.process(self._mlist, msg, {}) del msg['subject'] msg['subject'] = 'message 2' self._handler.process(self._mlist, msg, {}) # There are no digests already being sent, but the ant mailing list # does have a digest mbox collecting messages. get_queue_messages('digest', expected_count=0) mailbox_path = os.path.join(self._mlist.data_path, 'digest.mmdf') self.assertGreater(os.path.getsize(mailbox_path), 0) self._command.invoke(digests, ('-s', '-l', 'ant.example.com')) self._runner.run() # Now, there's no digest mbox and there's a plaintext digest in the # outgoing queue. self.assertFalse(os.path.exists(mailbox_path)) items = get_queue_messages('virgin', expected_count=1) digest_contents = str(items[0].msg) self.assertIn('Subject: message 1', digest_contents) self.assertIn('Subject: message 2', digest_contents) def test_send_one_digest_by_fqdn_listname(self): msg = mfs("""\ To: ant@example.com From: anne@example.com Subject: message 1 """) self._handler.process(self._mlist, msg, {}) del msg['subject'] msg['subject'] = 'message 2' self._handler.process(self._mlist, msg, {}) # There are no digests already being sent, but the ant mailing list # does have a digest mbox collecting messages. get_queue_messages('digest', expected_count=0) mailbox_path = os.path.join(self._mlist.data_path, 'digest.mmdf') self.assertGreater(os.path.getsize(mailbox_path), 0) self._command.invoke(digests, ('-s', '-l', 'ant@example.com')) self._runner.run() # Now, there's no digest mbox and there's a plaintext digest in the # outgoing queue. self.assertFalse(os.path.exists(mailbox_path)) items = get_queue_messages('virgin', expected_count=1) digest_contents = str(items[0].msg) self.assertIn('Subject: message 1', digest_contents) self.assertIn('Subject: message 2', digest_contents) def test_send_one_digest_to_missing_list_id(self): msg = mfs("""\ To: ant@example.com From: anne@example.com Subject: message 1 """) self._handler.process(self._mlist, msg, {}) del msg['subject'] msg['subject'] = 'message 2' self._handler.process(self._mlist, msg, {}) # There are no digests already being sent, but the ant mailing list # does have a digest mbox collecting messages. get_queue_messages('digest', expected_count=0) mailbox_path = os.path.join(self._mlist.data_path, 'digest.mmdf') self.assertGreater(os.path.getsize(mailbox_path), 0) result = self._command.invoke(digests, ('-s', '-l', 'bee.example.com')) self.assertEqual(result.exit_code, 0) self.assertEqual( result.output, 'No such list found: bee.example.com\n') self._runner.run() # And no digest was prepared. self.assertGreater(os.path.getsize(mailbox_path), 0) get_queue_messages('virgin', expected_count=0) def test_send_one_digest_to_missing_fqdn_listname(self): msg = mfs("""\ To: ant@example.com From: anne@example.com Subject: message 1 """) self._handler.process(self._mlist, msg, {}) del msg['subject'] msg['subject'] = 'message 2' self._handler.process(self._mlist, msg, {}) # There are no digests already being sent, but the ant mailing list # does have a digest mbox collecting messages. get_queue_messages('digest', expected_count=0) mailbox_path = os.path.join(self._mlist.data_path, 'digest.mmdf') self.assertGreater(os.path.getsize(mailbox_path), 0) result = self._command.invoke(digests, ('-s', '-l', 'bee@example.com')) self.assertEqual(result.exit_code, 0) self.assertEqual( result.output, 'No such list found: bee@example.com\n') self._runner.run() # And no digest was prepared. self.assertGreater(os.path.getsize(mailbox_path), 0) get_queue_messages('virgin', expected_count=0) def test_send_digest_to_one_missing_and_one_existing_list(self): msg = mfs("""\ To: ant@example.com From: anne@example.com Subject: message 1 """) self._handler.process(self._mlist, msg, {}) del msg['subject'] msg['subject'] = 'message 2' self._handler.process(self._mlist, msg, {}) # There are no digests already being sent, but the ant mailing list # does have a digest mbox collecting messages. get_queue_messages('digest', expected_count=0) mailbox_path = os.path.join(self._mlist.data_path, 'digest.mmdf') self.assertGreater(os.path.getsize(mailbox_path), 0) result = self._command.invoke( digests, ('-s', '-l', 'ant.example.com', '-l', 'bee.example.com')) self.assertEqual(result.exit_code, 0) self.assertEqual( result.output, 'No such list found: bee.example.com\n') self._runner.run() # But ant's digest was still prepared. self.assertFalse(os.path.exists(mailbox_path)) items = get_queue_messages('virgin', expected_count=1) digest_contents = str(items[0].msg) self.assertIn('Subject: message 1', digest_contents) self.assertIn('Subject: message 2', digest_contents) def test_send_digests_for_two_lists(self): # Populate ant's digest. msg = mfs("""\ To: ant@example.com From: anne@example.com Subject: message 1 """) self._handler.process(self._mlist, msg, {}) del msg['subject'] msg['subject'] = 'message 2' self._handler.process(self._mlist, msg, {}) # Create the second list. bee = create_list('bee@example.com') bee.digests_enabled = True bee.digest_size_threshold = 100000 bee.send_welcome_message = False member = subscribe(bee, 'Bart') member.preferences.delivery_mode = DeliveryMode.plaintext_digests # Populate bee's digest. msg = mfs("""\ To: bee@example.com From: bart@example.com Subject: message 3 """) self._handler.process(bee, msg, {}) del msg['subject'] msg['subject'] = 'message 4' self._handler.process(bee, msg, {}) # There are no digests for either list already being sent, but the # mailing lists do have a digest mbox collecting messages. ant_mailbox_path = os.path.join(self._mlist.data_path, 'digest.mmdf') self.assertGreater(os.path.getsize(ant_mailbox_path), 0) # Check bee's digest. bee_mailbox_path = os.path.join(bee.data_path, 'digest.mmdf') self.assertGreater(os.path.getsize(bee_mailbox_path), 0) # Both. get_queue_messages('digest', expected_count=0) # Process both list's digests. self._command.invoke( digests, ('-s', '-l', 'ant.example.com', '-l', 'bee@example.com')) self._runner.run() # Now, neither list has a digest mbox and but there are plaintext # digest in the outgoing queue for both. self.assertFalse(os.path.exists(ant_mailbox_path)) self.assertFalse(os.path.exists(bee_mailbox_path)) items = get_queue_messages('virgin', expected_count=2) # Figure out which digest is going to ant and which to bee. if items[0].msg['to'] == 'ant@example.com': ant = items[0].msg bee = items[1].msg else: assert items[0].msg['to'] == 'bee@example.com' ant = items[1].msg bee = items[0].msg # Check ant's digest. digest_contents = str(ant) self.assertIn('Subject: message 1', digest_contents) self.assertIn('Subject: message 2', digest_contents) # Check bee's digest. digest_contents = str(bee) self.assertIn('Subject: message 3', digest_contents) self.assertIn('Subject: message 4', digest_contents) def test_send_digests_for_all_lists(self): # Populate ant's digest. msg = mfs("""\ To: ant@example.com From: anne@example.com Subject: message 1 """) self._handler.process(self._mlist, msg, {}) del msg['subject'] msg['subject'] = 'message 2' self._handler.process(self._mlist, msg, {}) # Create the second list. bee = create_list('bee@example.com') bee.digests_enabled = True bee.digest_size_threshold = 100000 bee.send_welcome_message = False member = subscribe(bee, 'Bart') member.preferences.delivery_mode = DeliveryMode.plaintext_digests # Populate bee's digest. msg = mfs("""\ To: bee@example.com From: bart@example.com Subject: message 3 """) self._handler.process(bee, msg, {}) del msg['subject'] msg['subject'] = 'message 4' self._handler.process(bee, msg, {}) # There are no digests for either list already being sent, but the # mailing lists do have a digest mbox collecting messages. ant_mailbox_path = os.path.join(self._mlist.data_path, 'digest.mmdf') self.assertGreater(os.path.getsize(ant_mailbox_path), 0) # Check bee's digest. bee_mailbox_path = os.path.join(bee.data_path, 'digest.mmdf') self.assertGreater(os.path.getsize(bee_mailbox_path), 0) # Both. get_queue_messages('digest', expected_count=0) # Process all mailing list digests by not setting any arguments. self._command.invoke(digests, ('-s',)) self._runner.run() # Now, neither list has a digest mbox and but there are plaintext # digest in the outgoing queue for both. self.assertFalse(os.path.exists(ant_mailbox_path)) self.assertFalse(os.path.exists(bee_mailbox_path)) items = get_queue_messages('virgin', expected_count=2) # Figure out which digest is going to ant and which to bee. if items[0].msg['to'] == 'ant@example.com': ant = items[0].msg bee = items[1].msg else: assert items[0].msg['to'] == 'bee@example.com' ant = items[1].msg bee = items[0].msg # Check ant's digest. digest_contents = str(ant) self.assertIn('Subject: message 1', digest_contents) self.assertIn('Subject: message 2', digest_contents) # Check bee's digest. digest_contents = str(bee) self.assertIn('Subject: message 3', digest_contents) self.assertIn('Subject: message 4', digest_contents) def test_send_no_digest_ready(self): # If no messages have been sent through the mailing list, no digest # can be sent. mailbox_path = os.path.join(self._mlist.data_path, 'digest.mmdf') self.assertFalse(os.path.exists(mailbox_path)) self._command.invoke(digests, ('-s', '-l', 'ant.example.com')) self._runner.run() get_queue_messages('virgin', expected_count=0) def test_bump_before_send(self): self._mlist.digest_volume_frequency = DigestFrequency.monthly self._mlist.volume = 7 self._mlist.next_digest_number = 4 self._mlist.digest_last_sent_at = right_now() + timedelta( days=-32) msg = mfs("""\ To: ant@example.com From: anne@example.com Subject: message 1 """) self._handler.process(self._mlist, msg, {}) self._command.invoke( digests, ('-s', '--bump', '-l', 'ant.example.com')) self._runner.run() # The volume is 8 and the digest number is 2 because a digest was sent # after the volume/number was bumped. self.assertEqual(self._mlist.volume, 8) self.assertEqual(self._mlist.next_digest_number, 2) self.assertEqual(self._mlist.digest_last_sent_at, right_now()) items = get_queue_messages('virgin', expected_count=1) self.assertEqual(items[0].msg['subject'], 'Ant Digest, Vol 8, Issue 1') def test_send_periodic_one_by_listid(self): # Test sending digest using --periodic. msg = mfs("""\ To: ant@example.com From: anne@example.com Subject: message 1 """) self._handler.process(self._mlist, msg, {}) del msg['subject'] msg['subject'] = 'message 2' self._handler.process(self._mlist, msg, {}) # There are no digests already being sent, but the ant mailing list # does have a digest mbox collecting messages. get_queue_messages('digest', expected_count=0) mailbox_path = os.path.join(self._mlist.data_path, 'digest.mmdf') self.assertGreater(os.path.getsize(mailbox_path), 0) self._command.invoke(digests, ('-p', '-v', '-l', 'ant.example.com')) self._runner.run() # Now, there's no digest mbox and there's a plaintext digest in the # outgoing queue. self.assertFalse(os.path.exists(mailbox_path)) items = get_queue_messages('virgin', expected_count=1) digest_contents = str(items[0].msg) self.assertIn('Subject: message 1', digest_contents) self.assertIn('Subject: message 2', digest_contents) def test_send_periodic_set_false(self): # Test sending digest --periodic when the only Mailing List's # digest_send_periodic is set to false. # Test sending digest using --periodic. self._mlist.digest_send_periodic = False msg = mfs("""\ To: ant@example.com From: anne@example.com Subject: message 1 """) self._handler.process(self._mlist, msg, {}) # There are no digests already being sent, but the ant mailing list # does have a digest mbox collecting messages. get_queue_messages('digest', expected_count=0) mailbox_path = os.path.join(self._mlist.data_path, 'digest.mmdf') self.assertGreater(os.path.getsize(mailbox_path), 0) self._command.invoke(digests, ('-p', '-l', 'ant.example.com')) self._runner.run() # Now, even though the digest command was run, the mailbox should still # be there. self.assertTrue(os.path.exists(mailbox_path)) get_queue_messages('virgin', expected_count=0) def test_send_periodic_two_lists_one_set_false(self): # Test digests --periodic when one of the two lists has # digest_send_periodic set to false. # Populate ant's digest. self._mlist.digest_send_periodic = False msg = mfs("""\ To: ant@example.com From: anne@example.com Subject: message 1 """) self._handler.process(self._mlist, msg, {}) del msg['subject'] msg['subject'] = 'message 2' self._handler.process(self._mlist, msg, {}) # Create the second list. bee = create_list('bee@example.com') bee.digests_enabled = True bee.digest_size_threshold = 100000 bee.send_welcome_message = False member = subscribe(bee, 'Bart') member.preferences.delivery_mode = DeliveryMode.plaintext_digests # Populate bee's digest. msg = mfs("""\ To: bee@example.com From: bart@example.com Subject: message 3 """) self._handler.process(bee, msg, {}) del msg['subject'] msg['subject'] = 'message 4' self._handler.process(bee, msg, {}) # There are no digests for either list already being sent, but the # mailing lists do have a digest mbox collecting messages. ant_mailbox_path = os.path.join(self._mlist.data_path, 'digest.mmdf') self.assertGreater(os.path.getsize(ant_mailbox_path), 0) # Check bee's digest. bee_mailbox_path = os.path.join(bee.data_path, 'digest.mmdf') self.assertGreater(os.path.getsize(bee_mailbox_path), 0) # Both. get_queue_messages('digest', expected_count=0) # Process both list's digests. self._command.invoke( digests, ('-p', '-l', 'ant.example.com', '-l', 'bee@example.com')) self._runner.run() # Now, ant should still have it's mailbox file, but bee shouldn't. # Also, bee's message should be in the outgoing queue. self.assertTrue(os.path.exists(ant_mailbox_path)) self.assertFalse(os.path.exists(bee_mailbox_path)) items = get_queue_messages('virgin', expected_count=1) # Figure out which digest is going to ant and which to bee. assert items[0].msg['to'] == 'bee@example.com' # Check bee's digest. digest_contents = str(items[0].msg) self.assertIn('Subject: message 3', digest_contents) self.assertIn('Subject: message 4', digest_contents) class TestBumpVolume(unittest.TestCase): layer = ConfigLayer def setUp(self): self._mlist = create_list('ant@example.com') self._mlist.digest_volume_frequency = DigestFrequency.monthly self._mlist.volume = 7 self._mlist.next_digest_number = 4 self.right_now = right_now() self._command = CliRunner() def test_bump_one_list(self): self._mlist.digest_last_sent_at = self.right_now + timedelta( days=-32) self._command.invoke(digests, ('-b', '-l', 'ant.example.com')) self.assertEqual(self._mlist.volume, 8) self.assertEqual(self._mlist.next_digest_number, 1) self.assertEqual(self._mlist.digest_last_sent_at, self.right_now) def test_bump_two_lists(self): self._mlist.digest_last_sent_at = self.right_now + timedelta( days=-32) # Create the second list. bee = create_list('bee@example.com') bee.digest_volume_frequency = DigestFrequency.monthly bee.volume = 7 bee.next_digest_number = 4 bee.digest_last_sent_at = self.right_now + timedelta( days=-32) self._command.invoke( digests, ('-b', '-l', 'ant.example.com', '-l', 'bee.example.com')) self.assertEqual(self._mlist.volume, 8) self.assertEqual(self._mlist.next_digest_number, 1) self.assertEqual(self._mlist.digest_last_sent_at, self.right_now) def test_bump_verbose(self): result = self._command.invoke( digests, ('-v', '-b', '-l', 'ant.example.com')) self.assertMultiLineEqual(result.output, """\ ant.example.com is at volume 7, number 4 ant.example.com bumped to volume 7, number 5 """) def test_send_verbose(self): result = self._command.invoke( digests, ('-v', '-s', '-n', '-l', 'ant.example.com')) self.assertMultiLineEqual(result.output, """\ ant.example.com sent volume 7, number 4 """) class TestDigestCommand(unittest.TestCase): layer = ConfigLayer def setUp(self): self._mlist = create_list('ant@example.com') self._command = CliRunner() def test_send_and_periodic_options_exclusive(self): result = self._command.invoke(digests, ('-s', '-p', '-l', 'ant@example.com')) self.assertEqual(result.exit_code, 1) self.assertMultiLineEqual(result.output, """\ --send and --periodic flags cannot be used together """) mailman-3.2.2/src/mailman/commands/tests/test_cli_import.py0000644000175000017500000000652013442110351025171 0ustar maxkingmaxking00000000000000# Copyright (C) 2015-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Test the `mailman import21` subcommand.""" import unittest from click.testing import CliRunner from contextlib import ExitStack from importlib_resources import path from mailman.app.lifecycle import create_list from mailman.commands.cli_import import import21 from mailman.testing.layers import ConfigLayer from mailman.utilities.importer import Import21Error from pickle import dump from tempfile import NamedTemporaryFile from unittest.mock import patch class TestImport(unittest.TestCase): layer = ConfigLayer def setUp(self): self._command = CliRunner() self.mlist = create_list('ant@example.com') @patch('mailman.commands.cli_import.import_config_pck') def test_process_pickle_with_bounce_info(self, import_config_pck): # The sample data contains Mailman 2 bounce info, represented as # _BounceInfo instances. We throw these away when importing to # Mailman 3, but we have to fake the instance's classes, otherwise # unpickling the dictionaries will fail. with path('mailman.testing', 'config-with-instances.pck') as pckpath: pckfile = str(pckpath) try: self._command.invoke(import21, ('ant.example.com', pckfile)) except ImportError as error: self.fail('The pickle failed loading: {}'.format(error)) self.assertTrue(import_config_pck.called) def test_missing_list_spec(self): result = self._command.invoke(import21) self.assertEqual(result.exit_code, 2, result.output) self.assertEqual( result.output, 'Usage: import21 [OPTIONS] LISTSPEC PICKLE_FILE\n' 'Try "import21 --help" for help.\n\n' 'Error: Missing argument "LISTSPEC".\n') def test_pickle_with_nondict(self): with NamedTemporaryFile() as pckfile: with open(pckfile.name, 'wb') as fp: dump(['not', 'a', 'dict'], fp) result = self._command.invoke( import21, ('ant.example.com', pckfile.name)) self.assertIn('Ignoring non-dictionary', result.output) def test_pickle_with_bad_language(self): with ExitStack() as resources: pckfile = str(resources.enter_context( path('mailman.testing', 'config.pck'))) resources.enter_context( patch('mailman.utilities.importer.check_language_code', side_effect=Import21Error('Fake bad language code'))) result = self._command.invoke( import21, ('ant.example.com', pckfile)) self.assertIn('Fake bad language code', result.output) mailman-3.2.2/src/mailman/commands/tests/test_cli_inject.py0000644000175000017500000000573213442110351025137 0ustar maxkingmaxking00000000000000# Copyright (C) 2017-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Test the `inject` command.""" import unittest from click.testing import CliRunner from io import BytesIO, StringIO from mailman.app.lifecycle import create_list from mailman.commands.cli_inject import inject from mailman.testing.helpers import get_queue_messages from mailman.testing.layers import ConfigLayer test_msg = b"""\ To: ant@example.com From: user@example.com Message-ID: body """ class InterruptRaisingReader(StringIO): def read(self, count=None): # Fake enough of the API so click returns this instance unchanged. if count is None: raise KeyboardInterrupt return b'' class TestInject(unittest.TestCase): layer = ConfigLayer def setUp(self): self._command = CliRunner() create_list('ant@example.com') def test_inject_keyboard_interrupt(self): results = self._command.invoke( inject, ('-f', '-', 'ant.example.com'), input=InterruptRaisingReader()) self.assertEqual(results.exit_code, 1) self.assertEqual(results.output, 'Interrupted\n') def test_inject_no_such_list(self): result = self._command.invoke(inject, ('bee.example.com',)) self.assertEqual(result.exit_code, 2) self.assertEqual( result.output, 'Usage: inject [OPTIONS] LISTSPEC\n' 'Try "inject --help" for help.\n\n' 'Error: No such list: bee.example.com\n') def test_inject_no_such_queue(self): result = self._command.invoke( inject, ('--queue', 'bogus', 'ant.example.com')) self.assertEqual(result.exit_code, 2) self.assertEqual( result.output, 'Usage: inject [OPTIONS] LISTSPEC\n' 'Try "inject --help" for help.\n\n' 'Error: No such queue: bogus\n') def test_inject_no_filename_option(self): result = self._command.invoke( inject, (('ant.example.com',)), input=BytesIO(test_msg)) self.assertEqual(result.exit_code, 0) self.assertEqual(result.output, '') msg = get_queue_messages('in', expected_count=1)[0].msg # We can't compare the entire message because of inserted headers. self.assertEqual(msg.as_bytes()[:75], test_msg[:75]) mailman-3.2.2/src/mailman/commands/tests/test_cli_lists.py0000644000175000017500000000346313442110351025020 0ustar maxkingmaxking00000000000000# Copyright (C) 2015-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Additional tests for the `lists` command line subcommand.""" import unittest from click.testing import CliRunner from mailman.app.lifecycle import create_list from mailman.commands.cli_lists import lists from mailman.interfaces.domain import IDomainManager from mailman.testing.layers import ConfigLayer from zope.component import getUtility class TestLists(unittest.TestCase): layer = ConfigLayer def setUp(self): self._command = CliRunner() def test_lists_with_domain_option(self): # LP: #1166911 - non-matching lists were returned. getUtility(IDomainManager).add( 'example.net', 'An example domain.') create_list('test1@example.com') create_list('test2@example.com') # Only this one should show up. create_list('test3@example.net') create_list('test4@example.com') result = self._command.invoke(lists, ('--domain', 'example.net')) self.assertEqual(result.exit_code, 0) self.assertEqual( result.output, '1 matching mailing lists found:\ntest3@example.net\n') mailman-3.2.2/src/mailman/commands/tests/test_cli_members.py0000644000175000017500000001217513442110351025314 0ustar maxkingmaxking00000000000000# Copyright (C) 2015-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Test the `mailman members` command.""" import unittest from click.testing import CliRunner from mailman.app.lifecycle import create_list from mailman.commands.cli_members import members from mailman.interfaces.member import MemberRole from mailman.testing.helpers import subscribe from mailman.testing.layers import ConfigLayer from tempfile import NamedTemporaryFile class TestCLIMembers(unittest.TestCase): layer = ConfigLayer def setUp(self): self._mlist = create_list('ant@example.com') self._command = CliRunner() def test_no_such_list(self): result = self._command.invoke(members, ('bee.example.com',)) self.assertEqual(result.exit_code, 2) self.assertEqual( result.output, 'Usage: members [OPTIONS] LISTSPEC\n' 'Try "members --help" for help.\n\n' 'Error: No such list: bee.example.com\n') def test_role_administrator(self): subscribe(self._mlist, 'Anne', role=MemberRole.owner) subscribe(self._mlist, 'Bart', role=MemberRole.moderator) subscribe(self._mlist, 'Cate', role=MemberRole.nonmember) subscribe(self._mlist, 'Dave', role=MemberRole.member) with NamedTemporaryFile('w', encoding='utf-8') as outfp: self._command.invoke(members, ( '--role', 'administrator', '-o', outfp.name, 'ant.example.com')) with open(outfp.name, 'r', encoding='utf-8') as infp: lines = infp.readlines() self.assertEqual(len(lines), 2) self.assertEqual(lines[0], 'Anne Person \n') self.assertEqual(lines[1], 'Bart Person \n') def test_role_any(self): subscribe(self._mlist, 'Anne', role=MemberRole.owner) subscribe(self._mlist, 'Bart', role=MemberRole.moderator) subscribe(self._mlist, 'Cate', role=MemberRole.nonmember) subscribe(self._mlist, 'Dave', role=MemberRole.member) with NamedTemporaryFile('w', encoding='utf-8') as outfp: self._command.invoke(members, ( '--role', 'any', '-o', outfp.name, 'ant.example.com')) with open(outfp.name, 'r', encoding='utf-8') as infp: lines = infp.readlines() self.assertEqual(len(lines), 4) self.assertEqual(lines[0], 'Anne Person \n') self.assertEqual(lines[1], 'Bart Person \n') self.assertEqual(lines[2], 'Cate Person \n') self.assertEqual(lines[3], 'Dave Person \n') def test_role_moderator(self): subscribe(self._mlist, 'Anne', role=MemberRole.owner) subscribe(self._mlist, 'Bart', role=MemberRole.moderator) subscribe(self._mlist, 'Cate', role=MemberRole.nonmember) subscribe(self._mlist, 'Dave', role=MemberRole.member) with NamedTemporaryFile('w', encoding='utf-8') as outfp: self._command.invoke(members, ( '--role', 'moderator', '-o', outfp.name, 'ant.example.com')) with open(outfp.name, 'r', encoding='utf-8') as infp: lines = infp.readlines() self.assertEqual(len(lines), 1) self.assertEqual(lines[0], 'Bart Person \n') def test_role_nonmember(self): subscribe(self._mlist, 'Anne', role=MemberRole.owner) subscribe(self._mlist, 'Bart', role=MemberRole.moderator) subscribe(self._mlist, 'Cate', role=MemberRole.nonmember) subscribe(self._mlist, 'Dave', role=MemberRole.member) with NamedTemporaryFile('w', encoding='utf-8') as outfp: self._command.invoke(members, ( '--role', 'nonmember', '-o', outfp.name, 'ant.example.com')) with open(outfp.name, 'r', encoding='utf-8') as infp: lines = infp.readlines() self.assertEqual(len(lines), 1) self.assertEqual(lines[0], 'Cate Person \n') def test_already_subscribed_with_display_name(self): subscribe(self._mlist, 'Anne') with NamedTemporaryFile('w', buffering=1, encoding='utf-8') as infp: print('Anne Person ', file=infp) result = self._command.invoke(members, ( '--add', infp.name, 'ant.example.com')) self.assertEqual( result.output, 'Already subscribed (skipping): Anne Person \n' ) mailman-3.2.2/src/mailman/commands/tests/test_cli_qfile.py0000644000175000017500000000410713442110351024756 0ustar maxkingmaxking00000000000000# Copyright (C) 2017-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Test the qfile command.""" import unittest from click.testing import CliRunner from contextlib import ExitStack from mailman.commands.cli_qfile import qfile from mailman.testing.layers import ConfigLayer from pickle import dump from tempfile import NamedTemporaryFile from unittest.mock import patch class TestUnshunt(unittest.TestCase): layer = ConfigLayer maxDiff = None def setUp(self): self._command = CliRunner() def test_print_str(self): with NamedTemporaryFile() as tmp_qfile: with open(tmp_qfile.name, 'wb') as fp: dump('a simple string', fp) results = self._command.invoke(qfile, (tmp_qfile.name,)) self.assertEqual(results.output, """\ [----- start pickle -----] <----- start object 1 -----> a simple string [----- end pickle -----] """, results.output) def test_interactive(self): with ExitStack() as resources: tmp_qfile = resources.enter_context(NamedTemporaryFile()) mock = resources.enter_context(patch( 'mailman.commands.cli_qfile.interact')) with open(tmp_qfile.name, 'wb') as fp: dump('a simple string', fp) self._command.invoke(qfile, (tmp_qfile.name, '-i')) mock.assert_called_once_with( banner="Number of objects found (see the variable 'm'): 1") mailman-3.2.2/src/mailman/commands/tests/test_cli_shell.py0000644000175000017500000001646113442110351024773 0ustar maxkingmaxking00000000000000# Copyright (C) 2016-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Test the withlist/shell command.""" import os import unittest from click.testing import CliRunner from contextlib import ExitStack from mailman.app.lifecycle import create_list from mailman.commands.cli_withlist import shell from mailman.config import config from mailman.interfaces.usermanager import IUserManager from mailman.testing.helpers import configuration from mailman.testing.layers import ConfigLayer from mailman.utilities.modules import hacked_sys_modules from types import ModuleType from unittest.mock import MagicMock, patch try: import readline # noqa: F401 has_readline = True except ImportError: has_readline = False class TestShell(unittest.TestCase): layer = ConfigLayer maxDiff = None def setUp(self): self._command = CliRunner() def test_namespace(self): with patch('mailman.commands.cli_withlist.start_python') as mock: self._command.invoke(shell, ('--interactive',)) self.assertEqual(mock.call_count, 1) # Don't test that all names are available, just a few choice ones. positional, keywords = mock.call_args namespace = positional[0] self.assertIn('getUtility', namespace) self.assertIn('IArchiver', namespace) self.assertEqual(namespace['IUserManager'], IUserManager) @configuration('shell', banner='my banner') def test_banner(self): with patch('mailman.commands.cli_withlist.interact') as mock: self._command.invoke(shell, ('--interactive',)) self.assertEqual(mock.call_count, 1) positional, keywords = mock.call_args self.assertEqual(keywords['banner'], 'my banner\n') @unittest.skipUnless(has_readline, 'readline module is not available') @configuration('shell', history_file='$var_dir/history.py') def test_history_file(self): with patch('mailman.commands.cli_withlist.interact'): self._command.invoke(shell, ('--interactive',)) history_file = os.path.join(config.VAR_DIR, 'history.py') self.assertTrue(os.path.exists(history_file)) @configuration('shell', use_ipython='yes') def test_start_ipython4(self): mock = MagicMock() with hacked_sys_modules('IPython.terminal.embed', mock): self._command.invoke(shell, ('--interactive',)) posargs, kws = mock.InteractiveShellEmbed.instance().mainloop.call_args self.assertEqual( kws['display_banner'], 'Welcome to the GNU Mailman shell\n') @configuration('shell', use_ipython='yes') def test_start_ipython1(self): mock = MagicMock() with hacked_sys_modules('IPython.frontend.terminal.embed', mock): self._command.invoke(shell, ('--interactive',)) posargs, kws = mock.InteractiveShellEmbed.instance.call_args self.assertEqual( kws['banner1'], 'Welcome to the GNU Mailman shell\n') @configuration('shell', use_ipython='debug') def test_start_ipython_debug(self): mock = MagicMock() with hacked_sys_modules('IPython.terminal.embed', mock): self._command.invoke(shell, ('--interactive',)) posargs, kws = mock.InteractiveShellEmbed.instance().mainloop.call_args self.assertEqual( kws['display_banner'], 'Welcome to the GNU Mailman shell\n') @configuration('shell', use_ipython='oops') def test_start_ipython_invalid(self): mock = MagicMock() with hacked_sys_modules('IPython.terminal.embed', mock): results = self._command.invoke(shell, ('--interactive',)) self.assertEqual( results.output, 'Invalid value for [shell]use_python: oops\n') # mainloop() never got called. self.assertIsNone( mock.InteractiveShellEmbed.instance().mainloop.call_args) @configuration('shell', use_ipython='yes') def test_start_ipython_uninstalled(self): with ExitStack() as resources: # Pretend iPython isn't available at all. resources.enter_context(patch( 'mailman.commands.cli_withlist.start_ipython1', return_value=None)) resources.enter_context(patch( 'mailman.commands.cli_withlist.start_ipython4', return_value=None)) results = self._command.invoke(shell, ('--interactive',)) self.assertEqual( results.output, 'ipython is not available, set use_ipython to no\n') def test_regex_without_run(self): results = self._command.invoke(shell, ('-l', '^.*example.com')) self.assertEqual(results.exit_code, 2) self.assertEqual( results.output, 'Usage: shell [OPTIONS] [RUN_ARGS]...\n' 'Try "shell --help" for help.\n\n' 'Error: Regular expression requires --run\n') def test_listspec_without_run(self): create_list('ant@example.com') mock = MagicMock() with ExitStack() as resources: resources.enter_context( hacked_sys_modules('IPython.terminal.embed', mock)) interactive_mock = resources.enter_context(patch( 'mailman.commands.cli_withlist.do_interactive')) self._command.invoke(shell, ('-l', 'ant.example.com')) posargs, kws = interactive_mock.call_args self.assertEqual( posargs[1], "The variable 'm' is the ant.example.com mailing list") def test_listspec_without_run_no_such_list(self): results = self._command.invoke(shell, ('-l', 'ant.example.com')) self.assertEqual(results.exit_code, 2) self.assertEqual( results.output, 'Usage: shell [OPTIONS] [RUN_ARGS]...\n' 'Try "shell --help" for help.\n\n' 'Error: No such list: ant.example.com\n') def test_run_without_listspec(self): something = ModuleType('something') something.something = lambda: print('I am a something!') with hacked_sys_modules('something', something): results = self._command.invoke(shell, ('--run', 'something')) self.assertEqual(results.exit_code, 0) self.assertEqual(results.output, 'I am a something!\n') def test_run_bogus_listspec(self): results = self._command.invoke( shell, ('-l', 'bee.example.com', '--run', 'something')) self.assertEqual(results.exit_code, 2) self.assertEqual( results.output, 'Usage: shell [OPTIONS] [RUN_ARGS]...\n' 'Try "shell --help" for help.\n\n' 'Error: No such list: bee.example.com\n') mailman-3.2.2/src/mailman/commands/tests/test_cli_status.py0000644000175000017500000000433213442110351025201 0ustar maxkingmaxking00000000000000# Copyright (C) 2017-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Test the status command.""" import socket import unittest from click.testing import CliRunner from mailman.bin.master import WatcherState from mailman.commands.cli_status import status from mailman.testing.layers import ConfigLayer from unittest.mock import patch class FakeLock: details = ('localhost', 9999, None) class TestStatus(unittest.TestCase): layer = ConfigLayer maxDiff = None def setUp(self): self._command = CliRunner() def test_stale_lock(self): with patch('mailman.commands.cli_status.master_state', return_value=(WatcherState.stale_lock, FakeLock())): results = self._command.invoke(status) self.assertEqual(results.exit_code, WatcherState.stale_lock.value, results.output) self.assertEqual( results.output, 'GNU Mailman is stopped (stale pid: 9999)\n', results.output) def test_unknown_state(self): with patch('mailman.commands.cli_status.master_state', return_value=(WatcherState.host_mismatch, FakeLock())): results = self._command.invoke(status) self.assertEqual(results.exit_code, WatcherState.host_mismatch.value, results.output) self.assertEqual( results.output, 'GNU Mailman is in an unexpected state ' '(localhost != {})\n'.format(socket.getfqdn()), results.output) mailman-3.2.2/src/mailman/commands/tests/test_cli_unshunt.py0000644000175000017500000000310613442110351025360 0ustar maxkingmaxking00000000000000# Copyright (C) 2017-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Test the `unshunt` command.""" import unittest from click.testing import CliRunner from mailman.commands.cli_unshunt import unshunt from mailman.config import config from mailman.email.message import Message from mailman.testing.layers import ConfigLayer from unittest.mock import patch class TestUnshunt(unittest.TestCase): layer = ConfigLayer maxDiff = None def setUp(self): self._command = CliRunner() self._queue = config.switchboards['shunt'] def test_dequeue_fails(self): filebase = self._queue.enqueue(Message(), {}) with patch.object(self._queue, 'dequeue', side_effect=RuntimeError('oops!')): results = self._command.invoke(unshunt) self.assertEqual( results.output, 'Cannot unshunt message {}, skipping:\noops!\n'.format(filebase)) mailman-3.2.2/src/mailman/commands/tests/test_eml_confirm.py0000644000175000017500000001674413442110351025333 0ustar maxkingmaxking00000000000000# Copyright (C) 2012-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Test the `confirm` command.""" import unittest from mailman.app.lifecycle import create_list from mailman.commands.eml_confirm import Confirm from mailman.config import config from mailman.email.message import Message from mailman.interfaces.command import ContinueProcessing from mailman.interfaces.mailinglist import SubscriptionPolicy from mailman.interfaces.subscriptions import ISubscriptionManager from mailman.interfaces.usermanager import IUserManager from mailman.runners.command import CommandRunner, Results from mailman.testing.helpers import ( get_queue_messages, make_testable_runner, specialized_message_from_string as mfs, subscribe) from mailman.testing.layers import ConfigLayer from zope.component import getUtility class TestConfirmJoin(unittest.TestCase): """Test the `confirm` command when joining a mailing list.""" layer = ConfigLayer def setUp(self): self._mlist = create_list('test@example.com') anne = getUtility(IUserManager).create_address( 'anne@example.com', 'Anne Person') self._token, token_owner, member = ISubscriptionManager( self._mlist).register(anne) self._command = Confirm() # Clear the virgin queue. get_queue_messages('virgin') def test_welcome_message(self): # A confirmation causes a welcome message to be sent to the member, if # enabled by the mailing list. status = self._command.process( self._mlist, Message(), {}, (self._token,), Results()) self.assertEqual(status, ContinueProcessing.no) # There should be one messages in the queue; the welcome message. items = get_queue_messages('virgin', expected_count=1) # Grab the welcome message. welcome = items[0].msg self.assertEqual(welcome['subject'], 'Welcome to the "Test" mailing list') self.assertEqual(welcome['to'], 'Anne Person ') def test_no_welcome_message(self): # When configured not to send a welcome message, none is sent. self._mlist.send_welcome_message = False status = self._command.process( self._mlist, Message(), {}, (self._token,), Results()) self.assertEqual(status, ContinueProcessing.no) # There will be no messages in the queue. get_queue_messages('virgin', expected_count=0) def test_confim_token_twice(self): # Don't try to confirm the same token twice. # We test this by passing a result that already confirms the same # token and it doesn't try to look at the database. result = Results() result.confirms = self._token status = self._command.process( self._mlist, Message(), {}, (self._token,), result) self.assertEqual(status, ContinueProcessing.no) class TestConfirmLeave(unittest.TestCase): """Test the `confirm` command when leaving a mailing list.""" layer = ConfigLayer def setUp(self): self._mlist = create_list('test@example.com') anne = subscribe(self._mlist, 'Anne', email='anne@example.com') self._token, token_owner, member = ISubscriptionManager( self._mlist).unregister(anne.address) def test_confirm_leave(self): msg = mfs("""\ From: Anne Person To: test-confirm+{token}@example.com Subject: Re: confirm {token} """.format(token=self._token)) Confirm().process(self._mlist, msg, {}, (self._token,), Results()) # Anne is no longer a member of the mailing list. member = self._mlist.members.get_member('anne@example.com') self.assertIsNone(member) class TestEmailResponses(unittest.TestCase): """Test the `confirm` command through the command runner.""" layer = ConfigLayer def setUp(self): self._mlist = create_list('test@example.com') def test_confirm_then_moderate_workflow(self): # Issue #114 describes a problem when confirming the moderation email. self._mlist.subscription_policy = ( SubscriptionPolicy.confirm_then_moderate) bart = getUtility(IUserManager).create_address( 'bart@example.com', 'Bart Person') # Clear any previously queued confirmation messages. get_queue_messages('virgin') self._token, token_owner, member = ISubscriptionManager( self._mlist).register(bart) # There should now be one email message in the virgin queue, i.e. the # confirmation message sent to Bart. items = get_queue_messages('virgin', expected_count=1) msg = items[0].msg # Confirmations come first, so this one goes to the subscriber. self.assertEqual(msg['to'], 'bart@example.com') confirm, token = str(msg['subject']).split() self.assertEqual(confirm, 'confirm') self.assertEqual(token, self._token) # Craft a confirmation response with the expected tokens. user_response = Message() user_response['From'] = 'bart@example.com' user_response['To'] = 'test-confirm+{}@example.com'.format(token) user_response['Subject'] = 'Re: confirm {}'.format(token) user_response.set_payload('') # Process the message through the command runner. config.switchboards['command'].enqueue( user_response, listid='test.example.com') make_testable_runner(CommandRunner, 'command').run() # There are now two messages in the virgin queue. One is going to the # subscriber containing the results of their confirmation message, and # the other is to the moderators informing them that they need to # handle the moderation queue. items = get_queue_messages('virgin', expected_count=2) if items[0].msg['to'] == 'bart@example.com': results = items[0].msg moderator_msg = items[1].msg else: results = items[1].msg moderator_msg = items[0].msg # Check the moderator message first. self.assertEqual(moderator_msg['to'], 'test-owner@example.com') self.assertEqual( moderator_msg['subject'], 'New subscription request to Test from bart@example.com') lines = moderator_msg.get_payload().splitlines() self.assertEqual( lines[-2].strip(), 'For: Bart Person ') self.assertEqual(lines[-1].strip(), 'List: test@example.com') # Now check the results message. self.assertEqual( str(results['subject']), 'The results of your email commands') self.assertMultiLineEqual(results.get_payload(), """\ The results of your email command are provided below. - Original message details: From: bart@example.com Subject: Re: confirm {} Date: n/a Message-ID: n/a - Results: Confirmed - Done. """.format(token)) mailman-3.2.2/src/mailman/commands/tests/test_eml_help.py0000644000175000017500000000430113442110351024610 0ustar maxkingmaxking00000000000000# Copyright (C) 2012-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Additional tests for the `help` email command.""" import unittest from mailman.app.lifecycle import create_list from mailman.commands.eml_help import Help from mailman.email.message import Message from mailman.interfaces.command import ContinueProcessing from mailman.runners.command import Results from mailman.testing.layers import ConfigLayer class TestHelp(unittest.TestCase): """Test email help.""" layer = ConfigLayer def setUp(self): self._mlist = create_list('test@example.com') self._help = Help() def test_too_many_arguments(self): # Error message when too many help arguments are given. results = Results() status = self._help.process(self._mlist, Message(), {}, ('more', 'than', 'one'), results) self.assertEqual(status, ContinueProcessing.no) self.assertEqual(str(results), """\ The results of your email command are provided below. help: too many arguments: more than one """) def test_no_such_command(self): # Error message when asking for help on an existent command. results = Results() status = self._help.process(self._mlist, Message(), {}, ('doesnotexist',), results) self.assertEqual(status, ContinueProcessing.no) self.assertEqual(str(results), """\ The results of your email command are provided below. help: no such command: doesnotexist """) mailman-3.2.2/src/mailman/commands/tests/test_eml_membership.py0000644000175000017500000000410613442110351026016 0ustar maxkingmaxking00000000000000# Copyright (C) 2016-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Test the Leave command.""" import unittest from mailman.app.lifecycle import create_list from mailman.commands.eml_membership import Leave from mailman.email.message import Message from mailman.interfaces.mailinglist import SubscriptionPolicy from mailman.interfaces.usermanager import IUserManager from mailman.runners.command import Results from mailman.testing.helpers import set_preferred from mailman.testing.layers import ConfigLayer from zope.component import getUtility class TestLeave(unittest.TestCase): layer = ConfigLayer def setUp(self): self._mlist = create_list('ant@example.com') self._command = Leave() def test_confirm_leave_not_a_member(self): self._mlist.unsubscription_policy = SubscriptionPolicy.confirm # Try to unsubscribe someone who is not a member. Anne is a real # user, with a validated address, but she is not a member of the # mailing list. anne = getUtility(IUserManager).create_user('anne@example.com') set_preferred(anne) # Initiate an unsubscription. msg = Message() msg['From'] = 'anne@example.com' results = Results() self._command.process(self._mlist, msg, {}, (), results) self.assertEqual( str(results).splitlines()[-1], 'leave: anne@example.com is not a member of ant@example.com') mailman-3.2.2/src/mailman/compat/0000755000175000017500000000000013445614541017751 5ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/compat/__init__.py0000644000175000017500000000000013244427337022052 0ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/config/0000755000175000017500000000000013445614541017733 5ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/config/__init__.py0000644000175000017500000000156213442110351022033 0ustar maxkingmaxking00000000000000# Copyright (C) 2008-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Mailman configuration package.""" from mailman.config.config import Configuration from public import public public(config=Configuration()) mailman-3.2.2/src/mailman/config/alembic.cfg0000644000175000017500000000147213442110351021777 0ustar maxkingmaxking00000000000000# Copyright (C) 2014-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . [alembic] # Path to Alembic migration scripts. script_location: mailman.database:alembic mailman-3.2.2/src/mailman/config/config.py0000644000175000017500000003265713442110351021552 0ustar maxkingmaxking00000000000000# Copyright (C) 2006-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Configuration file loading and management.""" import os import sys import mailman.templates from configparser import ConfigParser from contextlib import ExitStack from flufl.lock import Lock from importlib_resources import path, read_text from lazr.config import ConfigSchema, as_boolean from mailman import version from mailman.interfaces.configuration import ( ConfigurationUpdatedEvent, IConfiguration, MissingConfigurationFileError) from mailman.interfaces.languages import ILanguageManager from mailman.utilities.filesystem import makedirs from mailman.utilities.modules import call_name, expand_path from public import public from string import Template from zope.component import getUtility from zope.event import notify from zope.interface import implementer SPACE = ' ' SPACERS = '\n' DIR_NAMES = ( 'archive', 'bin', 'cache', 'data', 'etc', 'list_data', 'lock', 'log', 'messages', 'queue', ) MAILMAN_CFG_TEMPLATE = """\ # AUTOMATICALLY GENERATED BY MAILMAN ON {} UTC # # This is your GNU Mailman 3 configuration file. You can edit this file to # configure Mailman to your needs, and Mailman will never overwrite it. # Additional configuration information is available here: # # https://mailman.readthedocs.io/en/latest/src/mailman/config/docs/config.html # # For example, uncomment the following lines to run Mailman in developer mode. # # [devmode] # enabled: yes # recipient: your.address@your.domain""" @public @implementer(IConfiguration) class Configuration: """The core global configuration object.""" def __init__(self): self.switchboards = {} self.QFILE_SCHEMA_VERSION = version.QFILE_SCHEMA_VERSION self._config = None self.filename = None # Whether to create run-time paths or not. This is for the test # suite, which will set this to False until the test layer is set up. self.create_paths = True # Create various registries. self.chains = {} self.rules = {} self.handlers = {} self.pipelines = {} self.commands = {} self.plugins = {} self.password_context = None self.db = None def _clear(self): """Clear the cached configuration variables.""" self.switchboards.clear() getUtility(ILanguageManager).clear() def __getattr__(self, name): """Delegate to the configuration object.""" return getattr(self._config, name) def __iter__(self): return iter(self._config) def load(self, filename=None): """Load the configuration from the schema and config files.""" with path('mailman.config', 'schema.cfg') as schema_file: schema = ConfigSchema(str(schema_file)) # If a configuration file was given, load it now too. First, load # the absolute minimum default configuration, then if a # configuration filename was given by the user, push it. with path('mailman.config', 'mailman.cfg') as config_path: self._config = schema.load(str(config_path)) if filename is None: self._post_process() else: self.filename = filename with open(filename, 'r', encoding='utf-8') as user_config: self.push(filename, user_config.read()) def push(self, config_name, config_string): """Push a new configuration onto the stack.""" self._clear() self._config.push(config_name, config_string) self._post_process() def pop(self, config_name): """Pop a configuration from the stack.""" self._clear() self._config.pop(config_name) self._post_process() def _post_process(self): """Perform post-processing after loading the configuration files.""" # Expand and set up all directories. self._expand_paths() self.ensure_directories_exist() notify(ConfigurationUpdatedEvent(self)) def _expand_paths(self): """Expand all configuration paths.""" # Set up directories. default_bin_dir = os.path.abspath(os.path.dirname(sys.executable)) # Now that we've loaded all the configuration files we're going to # load, set up some useful directories based on the settings in the # configuration file. layout = 'paths.' + self._config.mailman.layout for category in self._config.getByCategory('paths'): if category.name == layout: break else: print('No path configuration found:', layout, file=sys.stderr) sys.exit(1) # First, collect all variables in a substitution dictionary. $VAR_DIR # is taken from the environment or from the configuration file if the # environment is not set. Because the var_dir setting in the config # file could be a relative path, and because 'mailman start' chdirs to # $VAR_DIR, without this subprocesses bin/master and bin/runner will # create $VAR_DIR hierarchies under $VAR_DIR when that path is # relative. var_dir = os.environ.get('MAILMAN_VAR_DIR', category.var_dir) substitutions = dict( cwd=os.getcwd(), argv=default_bin_dir, var_dir=var_dir, template_dir=( os.path.dirname(mailman.templates.__file__) if category.template_dir == ':source:' else category.template_dir), ) # Directories. for name in DIR_NAMES: key = '{}_dir'.format(name) substitutions[key] = getattr(category, key) # Files. for name in ('lock', 'pid'): key = '{}_file'.format(name) substitutions[key] = getattr(category, key) # Add the path to the .cfg file, if one was given on the command line. if self.filename is not None: substitutions['cfg_file'] = self.filename # Now, perform substitutions recursively until there are no more # variables with $-vars in them, or until substitutions are not # helping any more. last_dollar_count = 0 while True: expandables = [] # Mutate the dictionary during iteration. for key in substitutions: raw_value = substitutions[key] value = Template(raw_value).safe_substitute(substitutions) if '$' in value: # Still more work to do. expandables.append((key, value)) substitutions[key] = value if len(expandables) == 0: break if len(expandables) == last_dollar_count: print('Path expansion infloop detected:\n', SPACERS.join('\t{}: {}'.format(key, value) for key, value in sorted(expandables)), file=sys.stderr) sys.exit(1) last_dollar_count = len(expandables) # Ensure that all paths are normalized and made absolute. Handle the # few special cases first. Most of these are due to backward # compatibility. self.PID_FILE = os.path.abspath(substitutions.pop('pid_file')) for key in substitutions: attribute = key.upper() setattr(self, attribute, os.path.abspath(substitutions[key])) @property def logger_configs(self): """Return all log config sections.""" return self._config.getByCategory('logging', []) @property def paths(self): """Return a substitution dictionary of all path variables.""" return dict((k, self.__dict__[k]) for k in self.__dict__ if k.endswith('_DIR')) def ensure_directories_exist(self): """Create all path directories if they do not exist.""" if self.create_paths: for variable, directory in self.paths.items(): makedirs(directory) # Avoid circular imports. from mailman.utilities.datetime import now # Create a mailman.cfg template file if it doesn't already exist. # LBYL: , but it's probably okay because the directories # likely didn't exist before the above loop, and we'll create a # temporary lock. lock_file = os.path.join(self.LOCK_DIR, 'mailman-cfg.lck') mailman_cfg = os.path.join(self.ETC_DIR, 'mailman.cfg') with Lock(lock_file): if not os.path.exists(mailman_cfg): with open(mailman_cfg, 'w') as fp: print(MAILMAN_CFG_TEMPLATE.format( now().replace(microsecond=0)), file=fp) @property def runner_configs(self): """Iterate over all the runner configuration sections.""" yield from self._config.getByCategory('runner', []) @property def archivers(self): """Iterate over all the archivers.""" for section in self._config.getByCategory('archiver', []): class_path = section['class'].strip() if len(class_path) == 0: continue archiver = call_name(class_path) archiver.is_enabled = as_boolean(section.enable) yield archiver @property def plugin_configs(self): """Return all the plugin configuration sections.""" plugin_sections = self._config.getByCategory('plugin', []) for section in plugin_sections: # 2017-08-27 barry: There's a fundamental constraint imposed by # lazr.config, namely that we have to use a .master section instead # of a .template section in the schema.cfg, or user supplied # configuration files cannot define new [plugin.*] sections. See # https://bugs.launchpad.net/lazr.config/+bug/310619 for # additional details. # # However, this means that [plugin.master] will show up in the # categories retrieved above. But 'master' is not a real plugin, # so we need to skip it (e.g. otherwise we'll get log warnings # about plugin.master being disabled, etc.). This imposes an # additional limitation though in that users cannot define a # plugin named 'master' because you can't override a master # section with a real section. There's no good way around this so # we just have to live with this limitation. if section.name == 'plugin.master': continue # The section.name will be something like 'plugin.example', but we # only want the 'example' part as the name of the plugin. We # could split on dots, but lazr.config gives us a different way. # `category_and_section_names` is a 2-tuple of e.g. # ('plugin', 'example'), so just grab the last element. yield section.category_and_section_names[1], section @property def language_configs(self): """Iterate over all the language configuration sections.""" yield from self._config.getByCategory('language', []) @public def load_external(path): """Load the configuration file named by path. :param path: A string naming the location of the external configuration file. This is either an absolute file system path or a special ``python:`` path. When path begins with ``python:``, the rest of the value must name a ``.cfg`` file located within Python's import path, however the trailing ``.cfg`` suffix is implied (don't provide it here). :return: The contents of the configuration file. :rtype: str """ # Is the context coming from a file system or Python path? if path.startswith('python:'): resource_path = path[7:] package, dot, resource = resource_path.rpartition('.') return read_text(package, resource + '.cfg') with open(path, 'r', encoding='utf-8') as fp: return fp.read() @public def external_configuration(path): """Parse the configuration file named by path. :param path: A string naming the location of the external configuration file. This is either an absolute file system path or a special ``python:`` path. When path begins with ``python:``, the rest of the value must name a ``.cfg`` file located within Python's import path, however the trailing ``.cfg`` suffix is implied (don't provide it here). :return: A `ConfigParser` instance. """ # Is the context coming from a file system or Python path? with ExitStack() as resources: cfg_path = str(expand_path(resources, path)) parser = ConfigParser() files = parser.read(cfg_path) if files != [cfg_path]: raise MissingConfigurationFileError(path) return parser mailman-3.2.2/src/mailman/config/configure.zcml0000644000175000017500000000743313244427337022614 0ustar maxkingmaxking00000000000000 mailman-3.2.2/src/mailman/config/docs/0000755000175000017500000000000013445614541020663 5ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/config/docs/config.rst0000644000175000017500000001362713442110351022656 0ustar maxkingmaxking00000000000000.. _configuration: ===================== Configuring Mailman ===================== Mailman is configured via an "ini"-style configuration file, usually called ``mailman.cfg``. Most of the defaults produce a usable system, but you will almost certainly have to set up a few things before you run Mailman for the first time. You only need to include those settings which you want to change; everything else is inherited. These file system paths are searched in the following order to find your site's custom ``mailman.cfg`` file. The first file found is used. * The file system path specified by the environment variable ``$MAILMAN_CONFIG_FILE`` * ``mailman.cfg`` in the current working directory * ``var/etc/mailman.cfg`` relative to the current working directory * ``$HOME/.mailman.cfg`` * ``/etc/mailman.cfg`` * ``/etc/mailman3/mailman.cfg`` * ``../../etc/mailman.cfg`` relative to the working directory of ``argv[0]`` You can also use the ``-C`` option to specify an explicit path, and this always takes precedence. See ``mailman --help`` for more details. You **must** restart Mailman for any changes to take effect. Which configuration file is in use? =================================== Mailman itself will tell you which configuration file is being used when you run the ``mailman info`` command:: $ mailman info GNU Mailman 3.1.0b4 (Between The Wheels) Python 3.5.3 (default, Jan 19 2017, 14:11:04) [GCC 6.3.0 20170118] config file: /home/mailman/var/etc/mailman.cfg db url: sqlite:////home/mailman/var/data/mailman.db devmode: DISABLED REST root url: http://localhost:8001/3.1/ REST credentials: restadmin:restpass The first time you run this command it will create the configuration file and directory using the built-in defaults, so use ``-C`` to specify an alternative location. Of course the ``info`` subcommand shows you other interesting things about your Mailman instance. Schemas, templates, and master sections ======================================= Mailman's configuration system is built on top of `lazr.config `_ although in general the details aren't important. Basically there is a ``schema.cfg`` file included in the source tree, which defines all the available sections and variables, along with global defaults. There is a built-in base ``mailman.cfg`` file also included in the source tree, which further refines the defaults. Your custom ``mailman.cfg`` file, found using the search locations described above, provides the final override for these settings. The ``schema.cfg`` file describes every section, variable, and permissible values, so you should consult this for more details. The ``schema.cfg`` file is included verbatim below. You will notice two types of special sections in the ``schema.cfg`` files; those that end with the ``.template`` suffix, and others which end in a ``.master`` suffix. There are no other special sections. Templates provide exactly that: a template for other similarly named sections. So for example, you will see a section labeled ``logging.template`` which provides some configuration variables and some basic defaults. You will also see a section called ``logging.bounce`` which refines the ``logging.template`` section by overriding one or more settings. If you wanted to change the default logging level for the database component in Mailman, say from ``warn`` to ``info``, you would add this to your ``mailman.cfg`` file:: [logging.database] level: info Generally you won't add new template specialization sections; everything you need is already defined. You will also see sections labeled with the ``.master`` suffix. For the most part you can treat these exactly the same as ``.template`` sections; the differences are only relevant for Mailman developers [#]_. An example of a ``.master`` section is ``[runner.master]`` which is used to define the defaults for all the :ref:`runner processes `. This is specialized in the built-in ``mailman.cfg`` file, where you'll see sections like ``[runner.archive]`` and ``[runner.in]``. You won't need to specially the master section yourself, but instead you can override some settings in the individual runner sections. How do I change a setting? ========================== If you think you want to change something, it can be a little tricky to find exactly the setting you'll need. The first step is to use the ``mailman conf`` command to print all the current variables and their values. With no options, this will print all the hundreds of (sorted!) available settings to standard output. You can narrow this down in two ways. You can print just the values of a particular section:: $ mailman conf -s webservice [webservice] admin_pass: restpass [webservice] admin_user: restadmin [webservice] api_version: 3.1 [webservice] hostname: localhost [webservice] port: 8001 [webservice] show_tracebacks: yes [webservice] use_https: no Let's say you wanted to change the port the REST API listens on. Just add this to your ``mailman.cfg`` file:: [webservice] port: 8080 You can also search for a specific setting:: $ mailman conf -k prompt [shell] prompt: >>> The ``mailman conf`` command does not provide documentation about sections or variables. In order to get more information about what a particular variable controls, read the ``schema.cfg`` and built-in base ``mailman.cfg`` file. schema.cfg ========== ``schema.cfg`` defines the ini-file schema and contains documentation for every section and configuration variable. .. literalinclude:: ../schema.cfg mailman.cfg =========== Configuration settings provided in the built-in base ``mailman.cfg`` file overrides those provided in ``schema.cfg``. .. literalinclude:: ../mailman.cfg .. [#] The technical differences are described in the `lazr.config `_ package, upon which Mailman's configuration system is based. mailman-3.2.2/src/mailman/config/exim4.cfg0000644000175000017500000000020113244427337021435 0ustar maxkingmaxking00000000000000[exim4] # Additional configuration variables for the Exim MTA, version 4. # Exim doesn't need any additional configuration yet. mailman-3.2.2/src/mailman/config/mail_archive.cfg0000644000175000017500000000220613442110351023022 0ustar maxkingmaxking00000000000000# Copyright (C) 2008-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . # This is the configuration file for the mail-archive.com archiver [general] # The base url for the archiver. This is used to to calculate links to # individual messages in the archive. base_url: https://www.mail-archive.com/ # If the archiver works by getting a copy of the message, this is the address # to send the copy to. # # See: https://www.mail-archive.com/faq.html#newlist recipient: archive@mail-archive.com mailman-3.2.2/src/mailman/config/mailman.cfg0000644000175000017500000000471513442110351022024 0ustar maxkingmaxking00000000000000# Copyright (C) 2008-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . # This is the absolute bare minimum base configuration file. User supplied # configurations are pushed onto this. [paths.local] # Directories as specified in schema.cfg, putting most stuff in # /var/tmp/mailman [paths.dev] # Convenient development layout where everything is put in a directory above # where the mailman.cfg file lives. var_dir: $cfg_file/../.. [paths.here] # Layout where the var directory is put in the current working directory. var_dir: $cwd/var [paths.fhs] # Filesystem Hiearchy Standard 2.3 # http://www.pathname.com/fhs/pub/fhs-2.3.html bin_dir: /sbin var_dir: /var/lib/mailman queue_dir: /var/spool/mailman log_dir: /var/log/mailman lock_dir: /var/lock/mailman etc_dir: /etc ext_dir: /etc/mailman.d pid_file: /var/run/mailman/master.pid [language.en] [runner.archive] class: mailman.runners.archive.ArchiveRunner [runner.bad] class: mailman.runners.fake.BadRunner # The bad runner is just a placeholder for its switchboard. start: no [runner.bounces] class: mailman.runners.bounce.BounceRunner [runner.command] class: mailman.runners.command.CommandRunner [runner.in] class: mailman.runners.incoming.IncomingRunner [runner.lmtp] class: mailman.runners.lmtp.LMTPRunner path: [runner.nntp] class: mailman.runners.nntp.NNTPRunner [runner.out] class: mailman.runners.outgoing.OutgoingRunner [runner.pipeline] class: mailman.runners.pipeline.PipelineRunner [runner.rest] class: mailman.runners.rest.RESTRunner path: [runner.retry] class: mailman.runners.retry.RetryRunner sleep_time: 15m [runner.shunt] class: mailman.runners.fake.ShuntRunner # The shunt runner is just a placeholder for its switchboard. start: no [runner.virgin] class: mailman.runners.virgin.VirginRunner [runner.digest] class: mailman.runners.digest.DigestRunner mailman-3.2.2/src/mailman/config/mhonarc.cfg0000644000175000017500000000215413442110351022030 0ustar maxkingmaxking00000000000000# Copyright (C) 2008-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . # This is the configuration file for the MHonArc archiver [general] # The base url for the archiver. This is used to to calculate links to # individual messages in the archive. base_url: http://$hostname/archives/$fqdn_listname # If the archiver works by calling a command on the local machine, this is the # command to call. command: /usr/bin/mhonarc -outdir /path/to/archive/$listname -add mailman-3.2.2/src/mailman/config/passlib.cfg0000644000175000017500000000054713442110351022042 0ustar maxkingmaxking00000000000000[passlib] # This is the output of passlibs.apps.custom_app_context.to_string(). # See https://passlib.readthedocs.io/en/stable/index.html for details. schemes = sha512_crypt, sha256_crypt default = sha512_crypt sha256_crypt__min_rounds = 80000 sha512_crypt__min_rounds = 60000 admin__sha256_crypt__min_rounds = 160000 admin__sha512_crypt__min_rounds = 120000 mailman-3.2.2/src/mailman/config/postfix.cfg0000644000175000017500000000112513244427337022111 0ustar maxkingmaxking00000000000000[postfix] # Additional configuration variables for the postfix MTA. # This variable describe the program to use for regenerating the transport map # db file, from the associated plain text files. The file being updated will # be appended to this string (with a separating space), so it must be # appropriate for os.system(). postmap_command: /usr/sbin/postmap # This variable describes the type of transport maps that will be generated by # mailman to be used with postfix for LMTP transport. By default, it is set to # hash, but mailman also supports `regex` tables. transport_file_type: hash mailman-3.2.2/src/mailman/config/schema.cfg0000644000175000017500000007406613442110351021654 0ustar maxkingmaxking00000000000000# Copyright (C) 2008-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . # This is the GNU Mailman configuration schema. It defines the default # configuration options for the core system and plugins. It uses ini-style # formats under the lazr.config regime to define all system configuration # options. See for details. [mailman] # This address is the "site owner" address. Certain messages which must be # delivered to a human, but which can't be delivered to a list owner (e.g. a # bounce from a list owner), will be sent to this address. It should point to # a human. site_owner: changeme@example.com # This is the local-part of an email address used in the From field whenever a # message comes from some entity to which there is no natural reply recipient. # Mailman will append '@' and the host name of the list involved. This # address must not bounce and it must not point to a Mailman process. noreply_address: noreply # The default language for this server. default_language: en # Membership tests for posting purposes are usually performed by looking at a # set of headers, passing the test if any of their values match a member of # the list. Headers are checked in the order given in this variable. The # value From_ means to use the envelope sender. Field names are case # insensitive. This is a space separate list of headers. sender_headers: from from_ reply-to sender # Mail command processor will ignore mail command lines after designated max. email_commands_max_lines: 10 # Default length of time a pending request is live before it is evicted from # the pending database. pending_request_life: 3d # How long should files be saved before they are evicted from the cache? cache_life: 7d # Which paths.* file system layout to use. layout: here # Can MIME filtered messages be preserved by list owners? filtered_messages_are_preservable: no # How should text/html parts be converted to text/plain when the mailing list # is set to convert HTML to plaintext? This names a command to be called, # where the substitution variable $filename is filled in by Mailman, and # contains the path to the temporary file that the command should read from. # The command should print the converted text to stdout. html_to_plain_text_command: /usr/bin/lynx -dump $filename # Specify what characters are allowed in list names. Characters outside of # the class [-_.+=!$*{}~0-9a-z] matched case insensitively are never allowed, # but this specifies a subset as the only allowable characters. This must be # a valid character class regexp or the effect on list creation is # unpredictable. listname_chars: [-_.0-9a-z] # These hooks are deprecated, but are kept here so as not to break existing # configuration files. However, these hooks are not run. Define a plugin # instead. pre_hook: post_hook: # Plugin configuration section template. # # To add a plugin, instantiate this section (changing `master` to whatever # your plugin's name is), and define at least a `path` and a `class`. When # the plugin is loaded, its subpackages will be search for components matching # the following interfaces: # # - IChain for new chains # - ICliSubCommand - `mailman` subcommands # - IEmailCommand - new email commands # - IHandler for new handlers # - IPipeline for new pipelines # - IRule for new rules # - IStyle for new styles. # # See the IPlugin interface for more details. [plugin.master] # The full Python import path for you IPlugin implementing class. It is # required to provide this. class: # Whether to enable this plugin or not. enabled: no # Additional configuration file for this plugin. If the value starts with # `python:` it is a Python import path, in which case the value should not # include the trailing .cfg (although the file is required to have this # suffix). Without `python:`, it is a file system path, and must be an # absolute path, since no guarantees are made about the current working # directory. configuration: # Package (as a dotted Python import path) to search for components that this # plugin wants to add, such as ISTyles, IRules, etc. If not given, the # plugin's name is used. component_package: [shell] # `mailman shell` (also `withlist`) gives you an interactive prompt that you # can use to interact with an initialized and configured Mailman system. Use # --help for more information. This section allows you to configure certain # aspects of this interactive shell. # Customize the interpreter prompt. prompt: >>> # Banner to show on startup. banner: Welcome to the GNU Mailman shell # Use IPython as the shell, which must be found on the system. Valid values # are `no`, `yes`, and `debug` where the latter is equivalent to `yes` except # that any import errors will be displayed to stderr. use_ipython: no # Set this to allow for command line history if readline is available. This # can be as simple as $var_dir/history.py to put the file in the var directory. history_file: [paths.master] # Important directories for Mailman operation. These are defined here so that # different layouts can be supported. For example, a developer layout would # be different from a FHS layout. Most paths are based off the var_dir, and # often just setting that will do the right thing for all the other paths. # You might also have to set spool_dir though. # # Substitutions are allowed, but must be of the form $var where 'var' names a # configuration variable in the paths.* section. Substitutions are expanded # recursively until no more $-variables are present. Beware of infinite # expansion loops! # # This is the root of the directory structure that Mailman will use to store # its run-time data. var_dir: /var/tmp/mailman # This is where the Mailman queue files directories will be created. queue_dir: $var_dir/queue # This is the directory containing the Mailman 'runner' and 'master' commands # if set to the string '$argv', it will be taken as the directory containing # the 'mailman' command. bin_dir: $argv # All list-specific data. list_data_dir: $var_dir/lists # Directory where log files go. log_dir: $var_dir/logs # Directory for system-wide locks. lock_dir: $var_dir/locks # Directory for system-wide data. data_dir: $var_dir/data # Cache files. cache_dir: $var_dir/cache # Directory for configuration files and such. etc_dir: $var_dir/etc # Directory where the default IMessageStore puts its messages. messages_dir: $var_dir/messages # Directory for archive backends to store their messages in. Archivers should # create a subdirectory in here to store their files. archive_dir: $var_dir/archives # Root directory for site-specific template override files. template_dir: $var_dir/templates # There are also a number of paths to specific file locations that can be # defined. For these, the directory containing the file must already exist, # or be one of the directories created by Mailman as per above. # # This is where PID file for the master runner is stored. pid_file: $var_dir/master.pid # Lock file. lock_file: $lock_dir/master.lck [devmode] # Setting enabled to true enables certain safeguards and other behavior # changes that make developing Mailman easier. For example, it forces the # SMTP RCPT TO recipients to be a test address so that no messages are # accidentally sent to real addresses. enabled: no # Set this to an address to force the SMTP RCPT TO recipents when devmode is # enabled. This way messages can't be accidentally sent to real addresses. recipient: # This gets set by the testing layers so that the runner subprocesses produce # predictable dates and times. testing: no # Time-outs for starting up various test subprocesses, such as the LMTP and # REST servers. This is only used for the test suite, so if you're seeing # test failures, try increasing the wait time. wait: 60s [passwords] # Where can we find the passlib configuration file? The path can be either a # file system path or a Python import path. If the value starts with python: # then it is a Python import path, otherwise it is a file system path. File # system paths must be absolute since no guarantees are made about the current # working directory. Python paths should not include the trailing .cfg, which # the file must end with. configuration: python:mailman.config.passlib # When Mailman generates them, this is the default length of passwords. password_length: 8 [runner.master] # Define which runners, and how many of them, to start. # The full import path to the class for this runner. class: mailman.core.runner.Runner # The queue directory path that this runner scans. This is ignored for # runners that don't manage a queue directory. path: $QUEUE_DIR/$name # The number of parallel runners. This must be a power of 2. This is ignored # for runners that don't manage a queue directory. instances: 1 # Whether to start this runner or not. start: yes # The maximum number of restarts for this runner. When the runner exits # because of an error or other unexpected problem, it is automatically # restarted, until the maximum number of restarts has been reached. max_restarts: 10 # The sleep interval for the runner. It wakes up once every interval to # process the files in its slice of the queue directory. Some runners may # ignore this. sleep_time: 1s [database] # The class implementing the IDatabase. class: mailman.database.sqlite.SQLiteDatabase # Use this to set the Storm database engine URL. You generally have one # primary database connection for all of Mailman. List data and most rosters # will store their data in this database, although external rosters may access # other databases in their own way. This string supports standard # 'configuration' substitutions. url: sqlite:///$DATA_DIR/mailman.db debug: no [logging.template] # This defines various log settings. The options available are: # # - level -- Overrides the default level; this may be any of the # standard Python logging levels, case insensitive. # - format -- Overrides the default format string # - datefmt -- Overrides the default date format string # - path -- Overrides the default logger path. This may be a relative # path name, in which case it is relative to Mailman's LOG_DIR, # or it may be an absolute path name. You cannot change the # handler class that will be used. # - propagate -- Boolean specifying whether to propagate log message from this # logger to the root "mailman" logger. You cannot override # settings for the root logger. # # In this section, you can define defaults for all loggers, which will be # prefixed by 'mailman.'. Use subsections to override settings for specific # loggers. The names of the available loggers are: # # - archiver -- All archiver output # - bounce -- All bounce processing logs go here # - config -- Configuration issues # - database -- Database logging (SQLAlchemy and Alembic) # - debug -- Only used for development # - error -- All exceptions go to this log # - fromusenet -- Information related to the Usenet to Mailman gateway # - http -- Internal wsgi-based web interface # - locks -- Lock state changes # - mischief -- Various types of hostile activity # - plugins -- Plugin logs # - runner -- Runner process start/stops # - smtp -- SMTP activity # - subscribe -- Information about leaves/joins # - vette -- Message vetting information format: %(asctime)s (%(process)d) %(message)s datefmt: %b %d %H:%M:%S %Y propagate: no level: info path: mailman.log [logging.root] [logging.archiver] [logging.bounce] path: bounce.log [logging.config] [logging.database] level: warn [logging.debug] path: debug.log level: info [logging.error] [logging.fromusenet] [logging.http] [logging.locks] [logging.mischief] [logging.plugins] path: plugins.log [logging.runner] [logging.smtp] path: smtp.log # The smtp logger defines additional options for handling the logging of each # attempted delivery. These format strings specify what information is logged # for every message, every successful delivery, every refused delivery and # every recipient failure. To disable a status message, set the value to 'no' # (without the quotes). # # These template strings accept the following set of substitution # placeholders, if available. # # msgid -- the Message-ID of the message in question # listname -- the fully-qualified list name # sender -- the sender if available # recip -- the recipient address if available, or the number of # recipients being delivered to # size -- the approximate size of the message in bytes # seconds -- the number of seconds the operation took # refused -- the number of refused recipients # smtpcode -- the SMTP success or failure code # smtpmsg -- the SMTP success or failure message every: $msgid smtp to $listname for $recip recips, completed in $time seconds success: $msgid post to $listname from $sender, $size bytes refused: $msgid post to $listname from $sender, $size bytes, $refused failures failure: $msgid delivery to $recip failed with code $smtpcode, $smtpmsg [logging.subscribe] [logging.vette] [webservice] # The hostname at which admin web service resources are exposed. hostname: localhost # The port at which the admin web service resources are exposed. port: 8001 # Whether or not requests to the web service are secured through SSL. use_https: no # Whether or not to show tracebacks in an HTTP response for a request that # raised an exception. show_tracebacks: yes # The API version number for the current (highest) API. api_version: 3.1 # The administrative username. admin_user: restadmin # The administrative password. admin_pass: restpass [language.master] # Template for language definitions. The section name must be [language.xx] # where xx is the 2-character ISO code for the language. # The English name for the language. description: English (USA) # And the default character set for the language. charset: us-ascii # Whether the language is enabled or not. enabled: yes # Language charsets as imported from Mailman 2.1 defaults # Ref: http://www.lingoes.net/en/translator/langcode.htm [language.ar] description: Arabic charset: utf-8 enabled: yes [language.ast] description: Asturian charset: iso-8859-1 enabled: yes [language.ca] description: Catalan charset: utf-8 enabled: yes [language.cs] description: Czech charset: iso-8859-2 enabled: yes [language.da] description: Danish charset: iso-8859-1 enabled: yes [language.de] description: German charset: utf-8 enabled: yes [language.el] description: Greek charset: iso-8859-7 enabled: yes [language.es] description: Spanish charset: iso-8859-1 enabled: yes [language.et] description: Estonian charset: iso-8859-15 enabled: yes [language.eu] # Basque description: Euskara charset: iso-8859-15 enabled: yes [language.fi] description: Finnish charset: iso-8859-1 enabled: yes [language.fr] description: French charset: iso-8859-1 enabled: yes [language.gl] description: Galician charset: utf-8 enabled: yes [language.he] description: Hebrew charset: utf-8 enabled: yes [language.hr] description: Croatian charset: iso-8859-2 enabled: yes [language.hu] description: Hungarian charset: iso-8859-2 enabled: yes [language.ia] description: Interlingua charset: iso-8859-15 enabled: yes [language.it] description: Italian charset: iso-8859-1 enabled: yes [language.ja] description: Japanese charset: euc-jp enabled: yes [language.ko] description: Korean charset: euc-kr enabled: yes [language.lt] description: Lithuanian charset: iso-8859-13 enabled: yes [language.nl] description: Dutch charset: iso-8859-1 enabled: yes [language.no] description: Norwegian charset: iso-8859-1 enabled: yes [language.pl] description: Polish charset: iso-8859-2 enabled: yes [language.pt] description: Protuguese charset: iso-8859-1 enabled: yes [language.pt_BR] description: Protuguese (Brazil) charset: iso-8859-1 enabled: yes [language.ro] description: Romanian charset: iso-8859-2 enabled: yes [language.ru] description: Russian charset: koi8-r enabled: yes [language.sk] description: Slovak charset: utf-8 enabled: yes [language.sl] description: Slovenian charset: iso-8859-2 enabled: yes [language.sr] description: Serbian charset: utf-8 enabled: yes [language.sv] description: Swedish charset: iso-8859-1 enabled: yes [language.tr] description: Turkish charset: iso-8859-9 enabled: yes [language.uk] description: Ukrainian charset: utf-8 enabled: yes [language.vi] description: Vietnamese charset: utf-8 enabled: yes [language.zh_CN] description: Chinese charset: utf-8 enabled: yes [language.zh_TW] description: Chinese (Taiwan) charset: utf-8 enabled: yes [antispam] # This section defines basic antispam detection settings. # This value contains lines which specify RFC 822 headers in the email to # check for spamminess. Each line contains a `key: value` pair, where the key # is the header to check and the value is a Python regular expression to match # against the header's value. Multiple checks should be entered as multiline # value with leading spaces: # # header_checks: # X-Spam: (yes|maybe) # Authentication-Results: mail.example.com; dmarc=(fail|quarantine) # # The header value and regular expression are always matched # case-insensitively. header_checks: # The chain to jump to if any of the header patterns matches. This must be # the name of an existing chain such as 'discard', 'reject', 'hold', or # 'accept', otherwise 'hold' will be used. jump_chain: hold [mta] # The class defining the interface to the incoming mail transport agent. incoming: mailman.mta.postfix.LMTP # The callable implementing delivery to the outgoing mail transport agent. # This must accept three arguments, the mailing list, the message, and the # message metadata dictionary. outgoing: mailman.mta.deliver.deliver # How to connect to the outgoing MTA. If smtp_user and smtp_pass is given, # then Mailman will attempt to log into the MTA when making a new connection. smtp_host: localhost smtp_port: 25 smtp_user: smtp_pass: # Where the LMTP server listens for connections. Use 127.0.0.1 instead of # localhost for Postfix integration, because Postfix only consults DNS # (e.g. not /etc/hosts). lmtp_host: 127.0.0.1 lmtp_port: 8024 # Ceiling on the number of recipients that can be specified in a single SMTP # transaction. Set to 0 to submit the entire recipient list in one # transaction. max_recipients: 500 # Ceiling on the number of SMTP sessions to perform on a single socket # connection. Some MTAs have limits. Set this to 0 to do as many as we like # (i.e. your MTA has no limits). Set this to some number great than 0 and # Mailman will close the SMTP connection and re-open it after this number of # consecutive sessions. max_sessions_per_connection: 0 # Maximum number of simultaneous subthreads that will be used for SMTP # delivery. After the recipients list is chunked according to max_recipients, # each chunk is handed off to the SMTP server by a separate such thread. If # your Python interpreter was not built for threads, this feature is disabled. # You can explicitly disable it in all cases by setting max_delivery_threads # to 0. max_delivery_threads: 0 # How long should messages which have delivery failures continue to be # retried? After this period of time, a message that has failed recipients # will be dequeued and those recipients will never receive the message. delivery_retry_period: 5d # These variables control the format and frequency of VERP-like delivery for # better bounce detection. VERP is Variable Envelope Return Path, defined # here: # # http://cr.yp.to/proto/verp.txt # # This involves encoding the address of the recipient as Mailman knows it into # the envelope sender address (i.e. RFC 5321 MAIL FROM). Thus, no matter what # kind of forwarding the recipient has in place, should it eventually bounce, # we will receive an unambiguous notice of the bouncing address. # # However, we're technically only "VERP-like" because we're doing the envelope # sender encoding in Mailman, not in the MTA. We do require cooperation from # the MTA, so you must be sure your MTA can be configured for extended address # semantics. # # The first variable describes how to encode VERP envelopes. It must contain # these three string interpolations, which get filled in by Mailman: # # $bounces -- the list's -bounces robot address will be set here # $local -- the recipient address's local mailbox part will be set here # $domain -- the recipient address's domain name will be set here # # This example uses the default below. # # FQDN list address is: mylist@dom.ain # Recipient is: aperson@a.nother.dom # # The envelope sender will be mylist-bounces+aperson=a.nother.dom@dom.ain # # Note that your MTA /must/ be configured to deliver such an addressed message # to mylist-bounces! verp_delimiter: + verp_format: ${bounces}+${local}=${domain} # For nicer confirmation emails, use a VERP-like format which encodes the # confirmation cookie in the reply address. This lets us put a more user # friendly Subject: on the message, but requires cooperation from the MTA. # Format is like verp_format, but with the following substitutions: # # $address -- the list-confirm address # $cookie -- the confirmation cookie verp_confirm_format: $address+$cookie # This regular expression unambiguously decodes VERP addresses, which will be # placed in the To: (or other, depending on the MTA) header of the bounce # message by the bouncing MTA. Getting this right is critical -- and tricky. # Learn your Python regular expressions. It must define exactly three named # groups, `bounces`, `local` and `domain`, with the same definition as above. # It will be compiled case-insensitively. verp_regexp: ^(?P[^+]+?)\+(?P[^=]+)=(?P[^@]+)@.*$ # This is analogous to verp_regexp, but for splitting apart the # verp_confirm_format. MUAs have been observed that mung # # From: local_part@host # # into # # To: "local_part" # # when replying, so we skip everything up to '<' if any. verp_confirm_regexp: ^(.*<)?(?P[^+]+?)\+(?P[^@]+)@.*$ # Set this to 'yes' to enable VERP-like (more user friendly) confirmations. verp_confirmations: no # Another good opportunity is when regular delivery is personalized. Here # again, we're already incurring the performance hit for addressing each # individual recipient. Set this to 'yes' to enable VERPs on all personalized # regular deliveries (personalized digests aren't supported yet). verp_personalized_deliveries: no # And finally, we can VERP normal, non-personalized deliveries. However, # because it can be a significant performance hit, we allow you to decide how # often to VERP regular deliveries. This is the interval, in number of # messages, to do a VERP recipient address. The same variable controls both # regular and digest deliveries. Set to 0 to disable occasional VERPs, set to # 1 to VERP every delivery, or to some number > 1 for only occasional VERPs. verp_delivery_interval: 0 # VERP format and regexp for probe messages. verp_probe_format: $bounces+$token@$domain verp_probe_regexp: ^(?P[^+]+?)\+(?P[^@]+)@.*$ # Set this 'yes' to activate VERP probe for disabling by bounce. verp_probes: no # This is the maximum number of automatic responses sent to an address because # of -request messages or posting hold messages. This limit prevents response # loops between Mailman and misconfigured remote email robots. Mailman # already inhibits automatic replies to any message labeled with a header # "Precendence: bulk|list|junk". This is a fallback safety valve so it should # be set fairly high. Set to 0 for no limit (probably useful only for # debugging). max_autoresponses_per_day: 10 # Some list posts and mail to the -owner address may contain DomainKey or # DomainKeys Identified Mail (DKIM) signature headers . # Various list transformations to the message such as adding a list header or # footer or scrubbing attachments or even reply-to munging can break these # signatures. It is generally felt that these signatures have value, even if # broken and even if the outgoing message is resigned. However, some sites # may wish to remove these headers by setting this to 'yes'. remove_dkim_headers: no # Where can we find the mail server specific configuration file? The path can # be either a file system path or a Python import path. If the value starts # with python: then it is a Python import path, otherwise it is a file system # path. File system paths must be absolute since no guarantees are made about # the current working directory. Python paths should not include the trailing # .cfg, which the file must end with. configuration: python:mailman.config.postfix [bounces] # How often should the bounce runner process queued detected bounces? register_bounces_every: 15m [archiver.master] # To add new archivers, define a new section based on this one, overriding the # following values. # The class implementing the IArchiver interface. class: # Set this to 'yes' to enable the archiver. enable: no # Additional configuration for the archiver. The path can be either a file # system path or a Python import path. If the value starts with python: then # it is a Python import path, otherwise it is a file system path. File system # paths must be absolute since no guarantees are made about the current # working directory. Python paths should not include the trailing .cfg, which # the file must end with. configuration: changeme # When sending the message to the archiver, you have the option of # "clobbering" the Date: header, specifically to make it more sane. Some # archivers can't handle dates that are wildly off from reality. This does # not change the Date: header for any other delivery vector except this # specific archive. # # When the original Date header is clobbered, it will always be stored in # X-Original-Date. The new Date header will always be set to the date at # which the messages was received by the Mailman server, in UTC. # # Your options here are: # * never -- Leaves the original Date header alone. # * always -- Always override the Date header. # * maybe -- Override the Date only if it is outside the clobber_skew period. clobber_date: maybe clobber_skew: 1d [archiver.mhonarc] # This is the stock MHonArc archiver. class: mailman.archiving.mhonarc.MHonArc configuration: python:mailman.config.mhonarc [archiver.mail_archive] # This is the stock mail-archive.com archiver. class: mailman.archiving.mailarchive.MailArchive configuration: python:mailman.config.mail_archive [archiver.prototype] # This is a prototypical sample archiver. class: mailman.archiving.prototype.Prototype [styles] # The default style to apply if nothing else was requested. The value is the # name of an existing style. If no such style exists, no style will be # applied. default: legacy-default [digests] # Headers which should be kept in both RFC 1153 (plain) and MIME digests. RFC # 1153 also specifies these headers in this exact order, so order matters. # These are space separated and case insensitive. mime_digest_keep_headers: Date From To Cc Subject Message-ID Keywords In-Reply-To References Content-Type MIME-Version Content-Transfer-Encoding Precedence Reply-To Message List-Post plain_digest_keep_headers: Message Date From Subject To Cc Message-ID Keywords Content-Type [nntp] # Set these variables if you need to authenticate to your NNTP server for # Usenet posting or reading. Leave these blank if no authentication is # necessary. user: password: # Host and port of the NNTP server to connect to. Leave these blank to use # the default localhost:119. host: port: # This controls how headers must be cleansed in order to be accepted by your # NNTP server. Some servers like INN reject messages containing prohibited # headers, or duplicate headers. The NNTP server may reject the message for # other reasons, but there's little that can be programmatically done about # that. # # These headers (case ignored) are removed from the original message. This is # a whitespace separate list of headers. remove_headers: nntp-posting-host nntp-posting-date x-trace x-complaints-to xref date-received posted posting-version relay-version received # These headers are left alone, unless there are duplicates in the original # message. Any second and subsequent headers are rewritten to the second # named header (case preserved). This is a list of header pairs, one pair per # line. rewrite_duplicate_headers: To X-Original-To CC X-Original-CC Content-Transfer-Encoding X-Original-Content-Transfer-Encoding MIME-Version X-MIME-Version [dmarc] # RFC 7489 - Domain-based Message Authentication, Reporting, and Conformance. # https://en.wikipedia.org/wiki/DMARC # Parameters for DMARC DNS lookups. If you are seeing 'DNSException: Unable # to query DMARC policy ...' entries in your error log, you may need to adjust # these. # # The time to wait for a response from a name server before timeout. resolver_timeout: 3s # The total time to spend trying to get an answer to the DNS question. resolver_lifetime: 5s # A URL from which to retrieve the data for the algorithm that computes # Organizational Domains for DMARC policy lookup purposes. This can be # anything handled by the Python urllib.request.urlopen function. See # https://publicsuffix.org/list/ for info. org_domain_data_url: https://publicsuffix.org/list/public_suffix_list.dat # How long should the local suffix list be used before it's considered out of # date. After this amount of time a new list will be downloaded, but if it # can't be accessed, old data will still be used. cache_lifetime: 7d mailman-3.2.2/src/mailman/config/tests/0000755000175000017500000000000013445614541021075 5ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/config/tests/__init__.py0000644000175000017500000000000013244427337023176 0ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/config/tests/test_archivers.py0000644000175000017500000000342713442110351024465 0ustar maxkingmaxking00000000000000# Copyright (C) 2013-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Site-wide archiver configuration tests.""" import unittest from mailman.config import config from mailman.testing.helpers import configuration from mailman.testing.layers import ConfigLayer class TestArchivers(unittest.TestCase): layer = ConfigLayer def test_enabled(self): # By default, the testing configuration enables some archivers. archivers = {} for archiver in config.archivers: archivers[archiver.name] = archiver self.assertFalse(archivers['prototype'].is_enabled) self.assertTrue(archivers['mail-archive'].is_enabled) self.assertTrue(archivers['mhonarc'].is_enabled) @configuration('archiver.mhonarc', enable='no') def test_disabled(self): # We just disabled one of the archivers. archivers = {} for archiver in config.archivers: archivers[archiver.name] = archiver self.assertFalse(archivers['prototype'].is_enabled) self.assertTrue(archivers['mail-archive'].is_enabled) self.assertFalse(archivers['mhonarc'].is_enabled) mailman-3.2.2/src/mailman/config/tests/test_configuration.py0000644000175000017500000001354713442110351025352 0ustar maxkingmaxking00000000000000# Copyright (C) 2012-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Test the system-wide global configuration.""" import os import unittest from contextlib import ExitStack from importlib_resources import path from mailman.config.config import ( Configuration, external_configuration, load_external) from mailman.interfaces.configuration import ( ConfigurationUpdatedEvent, MissingConfigurationFileError) from mailman.testing.helpers import configuration, event_subscribers from mailman.testing.layers import ConfigLayer from tempfile import NamedTemporaryFile, TemporaryDirectory from unittest import mock class TestConfiguration(unittest.TestCase): layer = ConfigLayer def test_push_and_pop_trigger_events(self): # Pushing a new configuration onto the stack triggers a # post-processing event. events = [] def on_event(event): # noqa: E306 if isinstance(event, ConfigurationUpdatedEvent): # Record both the event and the top overlay. events.append(event.config.overlays[0].name) # Do two pushes, and then pop one of them. with event_subscribers(on_event): with configuration('test', _configname='first'): with configuration('test', _configname='second'): pass self.assertEqual(events, ['first', 'second', 'first']) def test_config_template_dir_is_source(self): # This test will leave a 'var' directory in the top-level source # directory. Be sure to clean it up. config = Configuration() with ExitStack() as resources: fp = resources.enter_context( NamedTemporaryFile('w', encoding='utf-8')) var_dir = resources.enter_context(TemporaryDirectory()) # Don't let the post-processing after the config.load() to put a # 'var' directory in the source tree's top level directory. print("""\ [paths.here] template_dir: :source: var_dir: {} """.format(var_dir), file=fp) fp.flush() config.load(fp.name) import mailman.templates self.assertEqual(config.TEMPLATE_DIR, os.path.dirname(mailman.templates.__file__)) class TestExternal(unittest.TestCase): """Test external configuration file loading APIs.""" def test_load_external_by_filename(self): with path('mailman.config', 'postfix.cfg') as filename: contents = load_external(str(filename)) self.assertEqual(contents[:9], '[postfix]') def test_load_external_by_path(self): contents = load_external('python:mailman.config.postfix') self.assertEqual(contents[:9], '[postfix]') def test_external_configuration_by_filename(self): with path('mailman.config', 'postfix.cfg') as filename: parser = external_configuration(str(filename)) self.assertEqual(parser.get('postfix', 'postmap_command'), '/usr/sbin/postmap') def test_external_configuration_by_path(self): parser = external_configuration('python:mailman.config.postfix') self.assertEqual(parser.get('postfix', 'postmap_command'), '/usr/sbin/postmap') def test_missing_configuration_file(self): with self.assertRaises(MissingConfigurationFileError) as cm: external_configuration('path:mailman.config.missing') self.assertEqual(cm.exception.path, 'path:mailman.config.missing') class TestConfigurationErrors(unittest.TestCase): layer = ConfigLayer def test_bad_path_layout_specifier(self): # Using a [mailman]layout name that doesn't exist is a fatal error. config = Configuration() with ExitStack() as resources: fp = resources.enter_context( NamedTemporaryFile('w', encoding='utf-8')) print("""\ [mailman] layout: nonesuch """, file=fp) fp.flush() # Suppress warning messages in the test output. Also, make sure # that the config.load() call doesn't break global state. resources.enter_context(mock.patch('sys.stderr')) resources.enter_context(mock.patch.object(config, '_clear')) cm = resources.enter_context(self.assertRaises(SystemExit)) config.load(fp.name) self.assertEqual(cm.exception.args, (1,)) def test_path_expansion_infloop(self): # A path expansion never completes because it references a non-existent # substitution variable. config = Configuration() with ExitStack() as resources: fp = resources.enter_context( NamedTemporaryFile('w', encoding='utf-8')) print("""\ [paths.here] log_dir: $nopath/log_dir """, file=fp) fp.flush() # Suppress warning messages in the test output. Also, make sure # that the config.load() call doesn't break global state. resources.enter_context(mock.patch('sys.stderr')) resources.enter_context(mock.patch.object(config, '_clear')) cm = resources.enter_context(self.assertRaises(SystemExit)) config.load(fp.name) self.assertEqual(cm.exception.args, (1,)) mailman-3.2.2/src/mailman/core/0000755000175000017500000000000013445614541017416 5ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/core/__init__.py0000644000175000017500000000000013244427337021517 0ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/core/api.py0000644000175000017500000000456013442110351020531 0ustar maxkingmaxking00000000000000# Copyright (C) 2016-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """REST web service API contexts.""" from lazr.config import as_boolean from mailman.config import config from mailman.interfaces.api import IAPI from public import public from uuid import UUID from zope.interface import implementer @public @implementer(IAPI) class API30: version = '3.0' version_info = (3, 0) @classmethod def path_to(cls, resource): """See `IAPI`.""" return '{}://{}:{}/{}/{}'.format( ('https' if as_boolean(config.webservice.use_https) else 'http'), config.webservice.hostname, config.webservice.port, cls.version, (resource[1:] if resource.startswith('/') else resource), ) @staticmethod def from_uuid(uuid): """See `IAPI`.""" return uuid.int @staticmethod def to_uuid(uuid): """See `IAPI`.""" if isinstance(uuid, UUID): return uuid return UUID(int=int(uuid)) @public @implementer(IAPI) class API31: version = '3.1' version_info = (3, 1) @classmethod def path_to(cls, resource): """See `IAPI`.""" return '{}://{}:{}/{}/{}'.format( ('https' if as_boolean(config.webservice.use_https) else 'http'), config.webservice.hostname, config.webservice.port, cls.version, (resource[1:] if resource.startswith('/') else resource), ) @staticmethod def from_uuid(uuid): """See `IAPI`.""" return uuid.hex @staticmethod def to_uuid(uuid): """See `IAPI`.""" if isinstance(uuid, UUID): return uuid return UUID(hex=uuid) mailman-3.2.2/src/mailman/core/chains.py0000644000175000017500000000725113442110351021225 0ustar maxkingmaxking00000000000000# Copyright (C) 2007-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Application support for chain processing.""" from mailman.config import config from mailman.interfaces.chain import IChain, LinkAction from mailman.utilities.modules import add_components from public import public @public def process(mlist, msg, msgdata, start_chain='default-posting-chain'): """Process the message through a chain. :param mlist: the IMailingList for this message. :param msg: The Message object. :param msgdata: The message metadata dictionary. :param start_chain: The name of the chain to start the processing with. """ # Set up some bookkeeping. chain_stack = [] msgdata['rule_hits'] = hits = [] msgdata['rule_misses'] = misses = [] # Find the starting chain and begin iterating through its links. chain = config.chains[start_chain] chain_iter = chain.get_links(mlist, msg, msgdata) # Loop until we've reached the end of all processing chains. while chain: # Iterate over all links in the chain. Do this outside a for-loop so # we can capture a chain's link iterator in mid-flight. This supports # the 'detour' link action. try: link = next(chain_iter) except StopIteration: # This chain is exhausted. Pop the last chain on the stack and # continue iterating through it. If there's nothing left on the # chain stack then we're completely finished processing. if len(chain_stack) == 0: return chain, chain_iter = chain_stack.pop() continue if link.rule.check(mlist, msg, msgdata): if link.rule.record: hits.append(link.rule.name) # The rule matched so run its action. if link.action is LinkAction.jump: chain = link.chain chain_iter = chain.get_links(mlist, msg, msgdata) continue elif link.action is LinkAction.detour: # Push the current chain so that we can return to it when # the next chain is finished. chain_stack.append((chain, chain_iter)) chain = link.chain chain_iter = chain.get_links(mlist, msg, msgdata) continue elif link.action is LinkAction.stop: # Stop all processing. return elif link.action is LinkAction.defer: # Just process the next link in the chain. pass elif link.action is LinkAction.run: link.function(mlist, msg, msgdata) else: raise AssertionError( 'Bad link action: {}'.format(link.action)) else: # The rule did not match; keep going. if link.rule.record: misses.append(link.rule.name) @public def initialize(): """Set up chains, both built-in and from the database.""" add_components('chains', IChain, config.chains) mailman-3.2.2/src/mailman/core/constants.py0000644000175000017500000000314513442110351021772 0ustar maxkingmaxking00000000000000# Copyright (C) 2006-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Various constants and enumerations.""" from mailman.config import config from mailman.interfaces.languages import ILanguageManager from mailman.interfaces.member import DeliveryMode, DeliveryStatus from mailman.interfaces.preferences import IPreferences from public import public from zope.component import getUtility from zope.interface import implementer @implementer(IPreferences) class SystemDefaultPreferences: """The default system preferences.""" acknowledge_posts = False hide_address = True receive_list_copy = True receive_own_postings = True delivery_mode = DeliveryMode.regular delivery_status = DeliveryStatus.enabled @property def preferred_language(self): """Return the system preferred language.""" return getUtility(ILanguageManager)[config.mailman.default_language] public(system_preferences=SystemDefaultPreferences()) mailman-3.2.2/src/mailman/core/docs/0000755000175000017500000000000013445614541020346 5ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/core/docs/__init__.py0000644000175000017500000000000013244427337022447 0ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/core/docs/chains.rst0000644000175000017500000002075213421245331022342 0ustar maxkingmaxking00000000000000====== Chains ====== When a new message is posted to a mailing list, Mailman uses a set of rule chains to decide whether the message gets accepted for posting, rejected, discarded, or held for moderator approval. There are a number of built-in chains available that act as end-points in the processing of messages. The Discard chain ================= The `discard` chain simply throws the message away. :: >>> chain = config.chains['discard'] >>> print(chain.name) discard >>> print(chain.description) Discard a message and stop processing. >>> mlist = create_list('test@example.com') >>> msg = message_from_string("""\ ... From: aperson@example.com ... To: test@example.com ... Subject: My first post ... Message-ID: ... ... An important message. ... """) >>> def print_msgid(event): ... print('{0}: {1}'.format( ... event.chain.name.upper(), event.msg.get('message-id', 'n/a'))) >>> from mailman.core.chains import process >>> from mailman.testing.helpers import event_subscribers >>> with event_subscribers(print_msgid): ... process(mlist, msg, {}, 'discard') DISCARD: The Reject chain ================ The `reject` chain bounces the message back to the original sender, and logs this action. :: >>> chain = config.chains['reject'] >>> print(chain.name) reject >>> print(chain.description) Reject/bounce a message and stop processing. >>> with event_subscribers(print_msgid): ... process(mlist, msg, {}, 'reject') REJECT: The bounce message is now sitting in the `virgin` queue. >>> from mailman.testing.helpers import get_queue_messages >>> qfiles = get_queue_messages('virgin') >>> len(qfiles) 1 >>> print(qfiles[0].msg.as_string()) Subject: My first post From: test-owner@example.com To: aperson@example.com ... [No bounce details are available] ... Content-Type: message/rfc822 MIME-Version: 1.0 From: aperson@example.com To: test@example.com Subject: My first post Message-ID: An important message. ... The Hold Chain ============== The `hold` chain places the message into the administrative request database and depending on the list's settings, sends a notification to both the original sender and the list moderators. :: >>> chain = config.chains['hold'] >>> print(chain.name) hold >>> print(chain.description) Hold a message and stop processing. >>> with event_subscribers(print_msgid): ... process(mlist, msg, {}, 'hold') HOLD: There are now two messages in the virgin queue, one to the list moderators and one to the original author. >>> qfiles = get_queue_messages('virgin', sort_on='to') >>> len(qfiles) 2 One of the message is addressed to the mailing list moderators, and the other is addressed to the original sender. >>> from operator import itemgetter >>> messages = sorted((item.msg for item in qfiles), ... key=itemgetter('to'), reverse=True) This one is addressed to the list moderators. >>> print(messages[0].as_string()) Subject: test@example.com post from aperson@example.com requires approval From: test-owner@example.com To: test-owner@example.com MIME-Version: 1.0 ... As list administrator, your authorization is requested for the following mailing list posting: List: test@example.com From: aperson@example.com Subject: My first post The message is being held because: N/A At your convenience, visit your dashboard to approve or deny the request. ... Content-Type: message/rfc822 MIME-Version: 1.0 From: aperson@example.com To: test@example.com Subject: My first post Message-ID: Message-ID-Hash: 4CMWUN6BHVCMHMDAOSJZ2Q72G5M32MWB X-Message-ID-Hash: 4CMWUN6BHVCMHMDAOSJZ2Q72G5M32MWB An important message. ... Content-Type: message/rfc822 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: confirm ... From: test-request@example.com ... If you reply to this message, keeping the Subject: header intact, Mailman will discard the held message. Do this if the message is spam. If you reply to this message and include an Approved: header with the list password in it, the message will be approved for posting to the list. The Approved: header can also appear in the first line of the body of the reply. ... This message is addressed to the sender of the message. >>> print(messages[1].as_string()) MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Subject: Your message to test@example.com awaits moderator approval From: test-bounces@example.com To: aperson@example.com ... Your mail to 'test@example.com' with the subject My first post Is being held until the list moderator can review it for approval. The message is being held because: N/A Either the message will get posted to the list, or you will receive notification of the moderator's decision. The Accept chain ================ The `accept` chain sends the message on the `pipeline` queue, where it will be processed and sent on to the list membership. :: >>> chain = config.chains['accept'] >>> print(chain.name) accept >>> print(chain.description) Accept a message. >>> with event_subscribers(print_msgid): ... process(mlist, msg, {}, 'accept') ACCEPT: >>> qfiles = get_queue_messages('pipeline') >>> len(qfiles) 1 >>> print(qfiles[0].msg.as_string()) From: aperson@example.com To: test@example.com Subject: My first post Message-ID: Message-ID-Hash: 4CMWUN6BHVCMHMDAOSJZ2Q72G5M32MWB X-Message-ID-Hash: 4CMWUN6BHVCMHMDAOSJZ2Q72G5M32MWB An important message. Run-time chains =============== We can also define chains at run time, and these chains can be mutated. Run-time chains are made up of links where each link associates both a rule and a `jump`. The rule is really a rule name, which is looked up when needed. The jump names a chain which is jumped to if the rule matches. There is one built-in posting chain. This is the default chain to use when no other input chain is defined for a mailing list. It runs through the default rules. >>> chain = config.chains['default-posting-chain'] >>> print(chain.name) default-posting-chain >>> print(chain.description) The built-in moderation chain. Once the sender is a member of the mailing list, the previously created message is innocuous enough that it should pass through all default rules. This message will end up in the `pipeline` queue. :: >>> from mailman.testing.helpers import subscribe >>> subscribe(mlist, 'Anne') >>> with event_subscribers(print_msgid): ... process(mlist, msg, {}) ACCEPT: >>> qfiles = get_queue_messages('pipeline') >>> len(qfiles) 1 >>> print(qfiles[0].msg.as_string()) From: aperson@example.com To: test@example.com Subject: My first post Message-ID: Message-ID-Hash: 4CMWUN6BHVCMHMDAOSJZ2Q72G5M32MWB X-Message-ID-Hash: 4CMWUN6BHVCMHMDAOSJZ2Q72G5M32MWB X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header An important message. In addition, the message metadata now contains lists of all rules that have hit and all rules that have missed. >>> dump_list(qfiles[0].msgdata['rule_hits']) *Empty* >>> dump_list(qfiles[0].msgdata['rule_misses']) administrivia approved banned-address dmarc-mitigation emergency implicit-dest loop max-recipients max-size member-moderation news-moderation no-senders no-subject nonmember-moderation suspicious-header mailman-3.2.2/src/mailman/core/docs/core.rst0000644000175000017500000000006013244427337022026 0ustar maxkingmaxking00000000000000==== Core ==== .. toctree:: :glob: ./* mailman-3.2.2/src/mailman/core/docs/runner.rst0000644000175000017500000000447313244427337022423 0ustar maxkingmaxking00000000000000======= Runners ======= The *runners* are the processes that perform long-running tasks, such as moving messages around the Mailman queues. Some runners don't manage queues, such as the LMTP and REST API handling runners. Each runner that manages a queue directory though, is responsible for a slice of the hash space. It processes all the files in its slice, sleeps a little while, then wakes up and runs through its queue files again. Basic architecture ================== The basic architecture of runner is implemented in the base class that all runners inherit from. This base class implements a ``.run()`` method that runs continuously in a loop until the ``.stop()`` method is called. >>> mlist = create_list('test@example.com') Here is a very simple derived runner class. Runners use a configuration section in the configuration files to determine run characteristics, such as the queue directory to use. Here we push a configuration section for the test runner. :: >>> config.push('test-runner', """ ... [runner.test] ... max_restarts: 1 ... """) >>> from mailman.core.runner import Runner >>> class TestableRunner(Runner): ... def _dispose(self, mlist, msg, msgdata): ... self.msg = msg ... self.msgdata = msgdata ... return False ... ... def _do_periodic(self): ... self.stop() ... ... def _snooze(self, filecnt): ... return >>> runner = TestableRunner('test') This runner doesn't do much except run once, storing the message and metadata on instance variables. >>> msg = message_from_string("""\ ... From: aperson@example.com ... To: test@example.com ... ... A test message. ... """) >>> switchboard = config.switchboards['test'] >>> filebase = switchboard.enqueue(msg, listid=mlist.list_id, ... foo='yes', bar='no') >>> runner.run() >>> print(runner.msg.as_string()) From: aperson@example.com To: test@example.com A test message. >>> dump_msgdata(runner.msgdata) _parsemsg: False bar : no foo : yes lang : en listid : test.example.com version : 3 XXX More of the Runner API should be tested. .. Clean up. >>> config.pop('test-runner') mailman-3.2.2/src/mailman/core/docs/switchboard.rst0000644000175000017500000001236313244427337023420 0ustar maxkingmaxking00000000000000The switchboard =============== The switchboard is subsystem that moves messages between queues. Each instance of a switchboard is responsible for one queue directory. >>> msg = message_from_string("""\ ... From: aperson@example.com ... To: _xtest@example.com ... ... A test message. ... """) Create a switchboard by giving its queue name and directory. >>> import os >>> queue_directory = os.path.join(config.QUEUE_DIR, 'test') >>> from mailman.core.switchboard import Switchboard >>> switchboard = Switchboard('test', queue_directory) >>> print(switchboard.name) test >>> switchboard.queue_directory == queue_directory True Here's a helper function for ensuring things work correctly. >>> def check_qfiles(directory=None): ... if directory is None: ... directory = queue_directory ... files = {} ... for qfile in os.listdir(directory): ... root, ext = os.path.splitext(qfile) ... files[ext] = files.get(ext, 0) + 1 ... if len(files) == 0: ... print('empty') ... for ext in sorted(files): ... print('{0}: {1}'.format(ext, files[ext])) Enqueing and dequeing --------------------- The message can be enqueued with metadata specified in the passed in dictionary. >>> filebase = switchboard.enqueue(msg) >>> check_qfiles() .pck: 1 To read the contents of a queue file, dequeue it. >>> msg, msgdata = switchboard.dequeue(filebase) >>> print(msg.as_string()) From: aperson@example.com To: _xtest@example.com A test message. >>> dump_msgdata(msgdata) _parsemsg: False version : 3 >>> check_qfiles() .bak: 1 To complete the dequeing process, removing all traces of the message file, finish it (without preservation). >>> switchboard.finish(filebase) >>> check_qfiles() empty When enqueing a file, you can provide additional metadata keys by using keyword arguments. >>> filebase = switchboard.enqueue(msg, {'foo': 1}, bar=2) >>> msg, msgdata = switchboard.dequeue(filebase) >>> switchboard.finish(filebase) >>> dump_msgdata(msgdata) _parsemsg: False bar : 2 foo : 1 version : 3 Keyword arguments override keys from the metadata dictionary. >>> filebase = switchboard.enqueue(msg, {'foo': 1}, foo=2) >>> msg, msgdata = switchboard.dequeue(filebase) >>> switchboard.finish(filebase) >>> dump_msgdata(msgdata) _parsemsg: False foo : 2 version : 3 Iterating over files -------------------- There are two ways to iterate over all the files in a switchboard's queue. Normally, queue files end in .pck (for 'pickle') and the easiest way to iterate over just these files is to use the .files attribute. >>> filebase_1 = switchboard.enqueue(msg, foo=1) >>> filebase_2 = switchboard.enqueue(msg, foo=2) >>> filebase_3 = switchboard.enqueue(msg, foo=3) >>> filebases = sorted((filebase_1, filebase_2, filebase_3)) >>> sorted(switchboard.files) == filebases True >>> check_qfiles() .pck: 3 You can also use the .get_files() method if you want to iterate over all the file bases for some other extension. >>> for filebase in switchboard.get_files(): ... msg, msgdata = switchboard.dequeue(filebase) >>> bakfiles = sorted(switchboard.get_files('.bak')) >>> bakfiles == filebases True >>> check_qfiles() .bak: 3 >>> for filebase in switchboard.get_files('.bak'): ... switchboard.finish(filebase) >>> check_qfiles() empty Recovering files ---------------- Calling .dequeue() without calling .finish() leaves .bak backup files in place. These can be recovered when the switchboard is instantiated. >>> filebase_1 = switchboard.enqueue(msg, foo=1) >>> filebase_2 = switchboard.enqueue(msg, foo=2) >>> filebase_3 = switchboard.enqueue(msg, foo=3) >>> for filebase in switchboard.files: ... msg, msgdata = switchboard.dequeue(filebase) ... # Don't call .finish() >>> check_qfiles() .bak: 3 >>> switchboard_2 = Switchboard('test', queue_directory, recover=True) >>> check_qfiles() .pck: 3 The files can be recovered explicitly. >>> for filebase in switchboard.files: ... msg, msgdata = switchboard.dequeue(filebase) ... # Don't call .finish() >>> check_qfiles() .bak: 3 >>> switchboard.recover_backup_files() >>> check_qfiles() .pck: 3 But the files will only be recovered at most three times before they are considered defective. In order to prevent mail bombs and loops, once this maximum is reached, the files will be preserved in the 'bad' queue. :: >>> for filebase in switchboard.files: ... msg, msgdata = switchboard.dequeue(filebase) ... # Don't call .finish() >>> check_qfiles() .bak: 3 >>> switchboard.recover_backup_files() >>> check_qfiles() empty >>> bad = config.switchboards['bad'] >>> check_qfiles(bad.queue_directory) .psv: 3 Clean up -------- >>> for file in os.listdir(bad.queue_directory): ... os.remove(os.path.join(bad.queue_directory, file)) >>> check_qfiles(bad.queue_directory) empty Queue slices ------------ XXX Add tests for queue slices. mailman-3.2.2/src/mailman/core/i18n.py0000644000175000017500000000426013442110351020534 0ustar maxkingmaxking00000000000000# Copyright (C) 2009-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Internationalization.""" import mailman.messages from flufl.i18n import PackageStrategy, registry from mailman.interfaces.configuration import ConfigurationUpdatedEvent from public import public public(_=None) @public def initialize(application=None): """Initialize the i18n subsystem. :param application: An optional `flufl.i18n.Application` instance to use as the translation context. This primarily exists to support the testing environment. :type application: `flufl.i18n.Application` """ global _ if application is None: strategy = PackageStrategy('mailman', mailman.messages) application = registry.register(strategy) _ = application._ @public def handle_ConfigurationUpdatedEvent(event): if isinstance(event, ConfigurationUpdatedEvent): _.default = event.config.mailman.default_language @public def format_reasons(reasons): """Translate and format hold and rejection reasons. :param reasons: A list of reasons from the rules that hit. Each reason is a string to be translated or a tuple consisting of a string with {} replacements and one or more replacement values. :returns: A list of the translated and formatted strings. """ new_reasons = [] for reason in reasons: if isinstance(reason, tuple): new_reasons.append(_(reason[0]).format(*reason[1:])) else: new_reasons.append(_(reason)) return new_reasons mailman-3.2.2/src/mailman/core/initialize.py0000644000175000017500000002137113442110351022120 0ustar maxkingmaxking00000000000000# Copyright (C) 2006-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Initialize all global state. Every entrance into the Mailman system, be it by command line, mail program, or cgi, must call the initialize function here in order for the system's global state to be set up properly. Typically this is called after command line argument parsing, since some of the initialization behavior is controlled by the command line arguments. """ import os import sys import logging import mailman.config.config import mailman.core.logging from importlib_resources import read_text from mailman.interfaces.database import IDatabaseFactory from public import public from zope.component import getUtility from zope.configuration import xmlconfig # The test infrastructure uses this to prevent the search and loading of any # existing configuration file. Otherwise the existence of say a # ~/.mailman.cfg file can break tests. INHIBIT_CONFIG_FILE = object() public(INHIBIT_CONFIG_FILE=INHIBIT_CONFIG_FILE) def search_for_configuration_file(): """Search the file system for a configuration file to use. This is only called if the -C command line argument was not given. """ config_path = os.getenv('MAILMAN_CONFIG_FILE') # Both None and the empty string are considered "missing". if config_path and os.path.exists(config_path): return os.path.abspath(config_path) # ./mailman.cfg config_path = os.path.abspath('mailman.cfg') if os.path.exists(config_path): return config_path # As a special case, look in ./var/etc/mailman.cfg. We can't do this in # the Configuration.load() method because that depends on the # configuration system, which of course is not set up at that time! config_path = os.path.abspath(os.path.join('var', 'etc', 'mailman.cfg')) if os.path.exists(config_path): return config_path # ~/.mailman.cfg config_path = os.path.join(os.getenv('HOME', '~'), '.mailman.cfg') if os.path.exists(config_path): return os.path.abspath(config_path) # /etc/mailman.cfg config_path = '/etc/mailman.cfg' if os.path.exists(config_path): return os.path.abspath(config_path) # /etc/mailman3/mailman.cfg config_path = '/etc/mailman3/mailman.cfg' if os.path.exists(config_path): return os.path.abspath(config_path) # $argv0/../../etc/mailman.cfg bindir = os.path.dirname(sys.argv[0]) parent = os.path.dirname(bindir) config_path = os.path.join(parent, 'etc', 'mailman.cfg') if os.path.exists(config_path): return os.path.abspath(config_path) # Are there any others we should search by default? return None # These initialization calls are separated for the testing framework, which # needs to do some internal calculations after config file loading and log # initialization, but before database initialization. Generally all other # code will just call initialize(). @public def initialize_1(config_path=None): """First initialization step. * Zope component architecture * The configuration system * Run-time directories :param config_path: The path to the configuration file. :type config_path: string """ xmlconfig.string(read_text('mailman.config', 'configure.zcml')) # By default, set the umask so that only owner and group can read and # write our files. Specifically we must have g+rw and we probably want # o-rwx although I think in most cases it doesn't hurt if other can read # or write the files. os.umask(0o007) # Initialize configuration event subscribers. This must be done before # setting up the configuration system. from mailman.app.events import initialize as initialize_events initialize_events() # config_path will be set if the command line argument -C is given. That # case overrides all others. When not given on the command line, the # configuration file is searched for in the file system. if config_path is None: config_path = search_for_configuration_file() elif config_path is INHIBIT_CONFIG_FILE: # For the test suite, force this back to not using a config file. config_path = None mailman.config.config.load(config_path) # Use this environment variable to define an extra configuration file for # testing. This is used by the tox.ini to run the full test suite under # PostgreSQL. extra_cfg_path = os.environ.get('MAILMAN_EXTRA_TESTING_CFG') if extra_cfg_path is not None: with open(extra_cfg_path, 'r', encoding='utf-8') as fp: extra_cfg = fp.read() mailman.config.config.push('extra testing config', extra_cfg) @public def initialize_2(debug=False, propagate_logs=None, testing=False): """Second initialization step. * Database * Logging * Plugin pre_hook()'s * Rules * Chains * Pipelines * Commands :param debug: Should the database layer be put in debug mode? :type debug: boolean :param propagate_logs: Should the log output propagate to stderr? :type propagate_logs: boolean or None """ # Create the queue and log directories if they don't already exist. mailman.core.logging.initialize(propagate_logs) # Initialize plugins from mailman.plugins.initialize import initialize as initialize_plugins initialize_plugins() # Check for deprecated features in config. config = mailman.config.config if len(config.mailman.pre_hook) > 0: # pragma: nocover log = logging.getLogger('mailman.plugins') log.warning( 'The [mailman]pre_hook configuration value has been replaced ' "by the plugins infrastructure, and won't be called.") # Run the plugin pre_hooks, if one fails, disable the offending plugin. for name in config.plugins: # pragma: nocover plugin = config.plugins[name] if hasattr(plugin, 'pre_hook'): try: plugin.pre_hook() except Exception: # pragma: nocover log = logging.getLogger('mailman.plugins') log.exception('Plugin failed to run its pre_hook: {}' 'It will be disabled and its components ' "won't be loaded.".format(name)) del config.plugins[name] # Instantiate the database class, ensure that it's of the right type, and # initialize it. Then stash the object on our configuration object. utility_name = ('testing' if testing else 'production') config.db = getUtility(IDatabaseFactory, utility_name).create() # Initialize the rules and chains. Do the imports here so as to avoid # circular imports. from mailman.app.commands import initialize as initialize_commands from mailman.core.chains import initialize as initialize_chains from mailman.core.pipelines import initialize as initialize_pipelines from mailman.core.rules import initialize as initialize_rules # Order here is somewhat important. initialize_rules() initialize_chains() initialize_pipelines() initialize_commands() @public def initialize_3(): """Third initialization step. * Plugin post_hook()'s. """ # Run the plugin post_hooks. config = mailman.config.config log = logging.getLogger('mailman.plugins') if len(config.mailman.post_hook) > 0: # pragma: nocover log.warning( 'The [mailman]post_hook configuration value has been replaced ' "by the plugins infrastructure, and won't be called.") for plugin in config.plugins.values(): # pragma: nocover try: plugin.post_hook() except Exception: # pragma: nocover # A post_hook may fail, here we just hope for the best that the # plugin can work even if it post_hook failed as it's components # are already loaded. log.exception( 'Plugin failed to run its post_hook: {}'.format(plugin.name)) @public def initialize(config_path=None, propagate_logs=None): initialize_1(config_path) initialize_2(propagate_logs=propagate_logs) initialize_3() mailman-3.2.2/src/mailman/core/logging.py0000644000175000017500000001555213442110351021411 0ustar maxkingmaxking00000000000000# Copyright (C) 2006-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Logging initialization, using Python's standard logging package.""" import os import sys import stat import codecs import logging from lazr.config import as_boolean, as_log_level from mailman.config import config from public import public _handlers = {} # XXX I would love to simplify things and use Python's WatchedFileHandler, but # there are two problems. First, it's more difficult to handle the test # suite's need to reopen the file handler to a different path. # # The other problem is that WatchedFileHandler doesn't really easily support # HUPing the process to reopen the log file. Now, maybe that's not a big deal # because the standard logging module would already handle things correctly if # the file is moved, but still that's not an interface I'm ready to give up on # yet. For now, keep our hack. class ReopenableFileHandler(logging.Handler): """A file handler that supports reopening.""" def __init__(self, name, filename): super().__init__() self.name = name self.filename = filename self._stream = self._open() def _open(self): open_mode = 'a' try: status = os.stat(self.filename) if stat.S_ISREG(status.st_mode): open_mode = 'a' elif stat.S_ISFIFO(status.st_mode) or stat.S_ISCHR(status.st_mode): open_mode = 'w' except FileNotFoundError: open_mode = 'a' # Assume regular file return codecs.open(self.filename, open_mode, 'utf-8') def flush(self): if self._stream: self._stream.flush() def emit(self, record): # It's possible for the stream to have been closed by the time we get # here, due to the shut down semantics. This mostly happens in the # test suite, but be defensive anyway. stream = (self._stream if self._stream else sys.stderr) try: msg = self.format(record) try: stream.write('{}'.format(msg)) except UnicodeError: stream.write('{}'.format(msg.encode('string-escape'))) if msg[-1] != '\n': stream.write('\n') self.flush() except: # noqa: E722 pragma: nocover self.handleError(record) def close(self): self.flush() self._stream.close() self._stream = None super().close() def reopen(self, filename=None): """Reopen the output stream. :param filename: If given, this reopens the output stream to a new file. This is used in the test suite. :type filename: string """ if filename is not None: self.filename = filename self._stream.close() self._stream = self._open() def _init_logger(propagate, sub_name, log, logger_config): # Get settings from log configuration file (or defaults). log_format = logger_config.format log_datefmt = logger_config.datefmt # Propagation to the root logger is how we handle logging to stderr when # the runners are not run as a subprocess of 'mailman start'. log.propagate = (as_boolean(logger_config.propagate) if propagate is None else propagate) # Set the logger's level. log.setLevel(as_log_level(logger_config.level)) # Create a formatter for this logger, then a handler, and link the # formatter to the handler. formatter = logging.Formatter(fmt=log_format, datefmt=log_datefmt) path_str = logger_config.path path_abs = os.path.normpath(os.path.join(config.LOG_DIR, path_str)) handler = ReopenableFileHandler(sub_name, path_abs) _handlers[sub_name] = handler handler.setFormatter(formatter) log.addHandler(handler) @public def initialize(propagate=None): """Initialize all logs. :param propagate: Flag specifying whether logs should propagate their messages to the root logger. If omitted, propagation is determined from the configuration files. :type propagate: bool or None """ # First, find the root logger and configure the logging subsystem. # Initialize the root logger, then create a formatter for all the # sublogs. The root logger should log to stderr. logging.basicConfig(format=config.logging.root.format, datefmt=config.logging.root.datefmt, level=as_log_level(config.logging.root.level)) # Create the sub-loggers. Note that we'll redirect flufl.lock to # mailman.locks. for logger_config in config.logger_configs: sub_name = logger_config.name.split('.')[-1] if sub_name == 'root': continue if sub_name == 'locks': log = logging.getLogger('flufl.lock') # Explicitly prevent flufl.lock from propagating its log messages # to its root logger, i.e. the console. log.propagate = False if sub_name == 'database': # Set both the SQLAlchemy and Alembic logs to the mailman.database # log configuration, essentially ignoring the alembic.cfg settings. # Do the SQLAlchemy one first, then let the Alembic one fall # through to the common code path. log = logging.getLogger('sqlalchemy') _init_logger(propagate, sub_name, log, logger_config) log = logging.getLogger('alembic') elif sub_name == 'smtp': log = logging.getLogger('mail.log') _init_logger(propagate, sub_name, log, logger_config) log = logging.getLogger('mailman.smtp') else: logger_name = 'mailman.' + sub_name log = logging.getLogger(logger_name) _init_logger(propagate, sub_name, log, logger_config) @public def reopen(): """Re-open all log files.""" for handler in _handlers.values(): handler.reopen() @public def get_handler(sub_name): """Return the handler associated with a named logger. :param sub_name: The logger name, sans the 'mailman.' prefix. :type sub_name: string :return: The file handler associated with the named logger. :rtype: `ReopenableFileHandler` """ return _handlers[sub_name] mailman-3.2.2/src/mailman/core/pipelines.py0000644000175000017500000000500113442110351021737 0ustar maxkingmaxking00000000000000# Copyright (C) 2008-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Application support for pipeline processing.""" import logging from mailman.app.bounces import bounce_message from mailman.config import config from mailman.interfaces.handler import IHandler from mailman.interfaces.pipeline import ( DiscardMessage, IPipeline, RejectMessage) from mailman.utilities.modules import add_components from public import public dlog = logging.getLogger('mailman.debug') vlog = logging.getLogger('mailman.vette') @public def process(mlist, msg, msgdata, pipeline_name='built-in'): """Process the message through the given pipeline. :param mlist: the IMailingList for this message. :param msg: The Message object. :param msgdata: The message metadata dictionary. :param pipeline_name: The name of the pipeline to process through. """ message_id = msg.get('message-id', 'n/a') pipeline = config.pipelines[pipeline_name] for handler in pipeline: dlog.debug('{} pipeline {} processing: {}'.format( message_id, pipeline_name, handler.name)) try: handler.process(mlist, msg, msgdata) except DiscardMessage as error: vlog.info( '{} discarded by "{}" pipeline handler "{}": {}'.format( message_id, pipeline_name, handler.name, error.message)) except RejectMessage as error: vlog.info( '{} rejected by "{}" pipeline handler "{}": {}'.format( message_id, pipeline_name, handler.name, str(error))) bounce_message(mlist, msg, error) @public def initialize(): """Initialize the pipelines.""" # Find all handlers in the registered plugins. add_components('handlers', IHandler, config.handlers) # Set up some pipelines. add_components('pipelines', IPipeline, config.pipelines) mailman-3.2.2/src/mailman/core/rules.py0000644000175000017500000000207113442110351021105 0ustar maxkingmaxking00000000000000# Copyright (C) 2007-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Various rule helpers""" from mailman.config import config from mailman.interfaces.rules import IRule from mailman.utilities.modules import add_components from public import public @public def initialize(): """Find and register all rules in all plugins.""" # Find rules in plugins. add_components('rules', IRule, config.rules) mailman-3.2.2/src/mailman/core/runner.py0000644000175000017500000003113113442110351021263 0ustar maxkingmaxking00000000000000# Copyright (C) 2001-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """The process runner base class.""" import time import signal import logging import traceback from contextlib import suppress from io import StringIO from lazr.config import as_boolean, as_timedelta from mailman.config import config from mailman.core.i18n import _ from mailman.core.logging import reopen from mailman.core.switchboard import Switchboard from mailman.interfaces.languages import ILanguageManager from mailman.interfaces.listmanager import IListManager from mailman.interfaces.runner import ( IRunner, RunnerCrashEvent, RunnerInterrupt) from mailman.utilities.string import expand from public import public from zope.component import getUtility from zope.event import notify from zope.interface import implementer dlog = logging.getLogger('mailman.debug') elog = logging.getLogger('mailman.error') rlog = logging.getLogger('mailman.runner') @public @implementer(IRunner) class Runner: is_queue_runner = True def __init__(self, name, slice=None): """Create a runner. :param slice: The slice number for this runner. This is passed directly to the underlying `ISwitchboard` object. This is ignored for runners that don't manage a queue. :type slice: int or None """ # Grab the configuration section. self.name = name section = getattr(config, 'runner.' + name) substitutions = config.paths substitutions['name'] = name numslices = int(section.instances) # Check whether the runner is queue runner or not; non-queue runner # should not have queue_directory or switchboard instance. if self.is_queue_runner: self.queue_directory = expand(section.path, None, substitutions) self.switchboard = Switchboard( name, self.queue_directory, slice, numslices, True) else: self.queue_directory = None self.switchboard = None self.sleep_time = as_timedelta(section.sleep_time) # sleep_time is a timedelta; turn it into a float for time.sleep(). self.sleep_float = (86400 * self.sleep_time.days + self.sleep_time.seconds + self.sleep_time.microseconds / 1.0e6) self.max_restarts = int(section.max_restarts) self.start = as_boolean(section.start) self._stop = False self.status = 0 def __repr__(self): return '<{} at {:#x}>'.format(self.__class__.__name__, id(self)) def signal_handler(self, signum, frame): # pragma: nocover signame = { signal.SIGTERM: 'SIGTERM', signal.SIGINT: 'SIGINT', signal.SIGUSR1: 'SIGUSR1', }.get(signum, signum) if signum == signal.SIGHUP: reopen() rlog.info('{} runner caught SIGHUP. Reopening logs.'.format( self.name)) elif signum in (signal.SIGTERM, signal.SIGINT, signal.SIGUSR1): self.stop() self.status = signum rlog.info('{} runner caught {}. Stopping.'.format( self.name, signame)) # As of Python 3.5, PEP 475 gets in our way. Runners with long # time.sleep()'s in their _snooze() method (e.g. the retry runner) # will have their system call implemented time.sleep() # automatically retried at the C layer. The only reliable way to # prevent this is to raise an exception in the signal handler. The # standard run() method automatically suppresses this exception, # meaning, it's caught and ignored, but effectively breaks the # run() loop, which is just what we want. Runners which implement # their own run() method must be prepared to catch # RunnerInterrupts, usually also ignoring them. raise RunnerInterrupt def set_signals(self): """See `IRunner`.""" signal.signal(signal.SIGHUP, self.signal_handler) signal.signal(signal.SIGINT, self.signal_handler) signal.signal(signal.SIGTERM, self.signal_handler) signal.signal(signal.SIGUSR1, self.signal_handler) def stop(self): """See `IRunner`.""" self._stop = True def run(self): """See `IRunner`.""" # Start the main loop for this runner. with suppress(KeyboardInterrupt, RunnerInterrupt): while True: # Once through the loop that processes all the files in the # queue directory. filecnt = self._one_iteration() # Do the periodic work for the subclass. self._do_periodic() # If the stop flag is set, we're done. if self._stop: break # Give the runner an opportunity to snooze for a while, but # pass it the file count so it can decide whether to do more # work now or not. self._snooze(filecnt) self._clean_up() def _one_iteration(self): """See `IRunner`.""" me = self.__class__.__name__ dlog.debug('[%s] starting oneloop', me) # List all the files in our queue directory. The switchboard is # guaranteed to hand us the files in FIFO order. files = self.switchboard.files for filebase in files: dlog.debug('[%s] processing filebase: %s', me, filebase) try: # Ask the switchboard for the message and metadata objects # associated with this queue file. msg, msgdata = self.switchboard.dequeue(filebase) except Exception as error: # This used to just catch email.Errors.MessageParseError, but # other problems can occur in message parsing, e.g. # ValueError, and exceptions can occur in unpickling too. We # don't want the runner to die, so we just log and skip this # entry, but preserve it for analysis. self._log(error) elog.error('Skipping and preserving unparseable message: %s', filebase) self.switchboard.finish(filebase, preserve=True) config.db.abort() continue try: dlog.debug('[%s] processing onefile', me) self._process_one_file(msg, msgdata) dlog.debug('[%s] finishing filebase: %s', me, filebase) self.switchboard.finish(filebase) except Exception as error: # All runners that implement _dispose() must guarantee that # exceptions are caught and dealt with properly. Still, there # may be a bug in the infrastructure, and we do not want those # to cause messages to be lost. Any uncaught exceptions will # cause the message to be stored in the shunt queue for human # intervention. self._log(error) # Put a marker in the metadata for unshunting. msgdata['whichq'] = self.switchboard.name # It is possible that shunting can throw an exception, e.g. a # permissions problem or a MemoryError due to a really large # message. Try to be graceful. try: shunt = config.switchboards['shunt'] new_filebase = shunt.enqueue(msg, msgdata) elog.error('SHUNTING: %s', new_filebase) self.switchboard.finish(filebase) except Exception as error: # The message wasn't successfully shunted. Log the # exception and try to preserve the original queue entry # for possible analysis. self._log(error) elog.error( 'SHUNTING FAILED, preserving original entry: %s', filebase) self.switchboard.finish(filebase, preserve=True) config.db.abort() # Other work we want to do each time through the loop. dlog.debug('[%s] doing periodic', me) self._do_periodic() dlog.debug('[%s] committing transaction', me) config.db.commit() dlog.debug('[%s] checking short circuit', me) if self._short_circuit(): dlog.debug('[%s] short circuiting', me) break dlog.debug('[%s] ending oneloop: %s', me, len(files)) return len(files) def _process_one_file(self, msg, msgdata): """See `IRunner`.""" # Do some common sanity checking on the message metadata. It's got to # be destined for a particular mailing list. This switchboard is used # to shunt off badly formatted messages. We don't want to just trash # them because they may be fixable with human intervention. Just get # them out of our sight. # # Find out which mailing list this message is destined for. mlist = None missing = object() # First try to dig out the target list by id. If there's no list-id # in the metadata, fall back to the fqdn list name for backward # compatibility. list_manager = getUtility(IListManager) list_id = msgdata.get('listid', missing) fqdn_listname = None if list_id is missing: fqdn_listname = msgdata.get('listname', missing) # XXX Deprecate. if fqdn_listname is not missing: mlist = list_manager.get(fqdn_listname) else: mlist = list_manager.get_by_list_id(list_id) if mlist is None: identifier = (list_id if list_id is not None else fqdn_listname) elog.error( '%s runner "%s" shunting message for missing list: %s', msg['message-id'], self.name, identifier) config.switchboards['shunt'].enqueue(msg, msgdata) return # Now process this message. We also want to set up the language # context for this message. The context will be the preferred # language for the user if the sender is a member of the list, or it # will be the list's preferred language. However, we must take # special care to reset the defaults, otherwise subsequent messages # may be translated incorrectly. if mlist is None: language_manager = getUtility(ILanguageManager) language = language_manager[config.mailman.default_language] elif msg.sender: member = mlist.members.get_member(msg.sender) language = (member.preferred_language if member is not None else mlist.preferred_language) else: language = mlist.preferred_language with _.using(language.code): msgdata['lang'] = language.code try: keepqueued = self._dispose(mlist, msg, msgdata) except Exception as error: # Trigger the Zope event and re-raise notify(RunnerCrashEvent(self, mlist, msg, msgdata, error)) raise if keepqueued: self.switchboard.enqueue(msg, msgdata) def _log(self, exc): elog.error('Uncaught runner exception: %s', exc) s = StringIO() traceback.print_exc(file=s) elog.error('%s', s.getvalue()) def _clean_up(self): """See `IRunner`.""" pass def _dispose(self, mlist, msg, msgdata): """See `IRunner`.""" raise NotImplementedError def _do_periodic(self): """See `IRunner`.""" pass def _snooze(self, filecnt): """See `IRunner`.""" if filecnt or self.sleep_float <= 0: return time.sleep(self.sleep_float) def _short_circuit(self): """See `IRunner`.""" return self._stop mailman-3.2.2/src/mailman/core/switchboard.py0000644000175000017500000002704513442110351022274 0ustar maxkingmaxking00000000000000# Copyright (C) 2001-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Queuing and dequeuing message/metadata pickle files. Messages are represented as email.message.Message objects (or an instance ofa subclass). Metadata is represented as a Python dictionary. For every message/metadata pair in a queue, a single file containing two pickles is written. First, the message is written to the pickle, then the metadata dictionary is written. """ import os import time import email import pickle import hashlib import logging from mailman.config import config from mailman.email.message import Message from mailman.interfaces.configuration import ConfigurationUpdatedEvent from mailman.interfaces.switchboard import ISwitchboard from mailman.utilities.filesystem import makedirs from mailman.utilities.string import expand from public import public from zope.interface import implementer # 20 bytes of all bits set, maximum hashlib.sha.digest() value. We do it this # way for Python 2/3 compatibility. shamax = int('0xffffffffffffffffffffffffffffffffffffffff', 16) # Small increment to add to time in case two entries have the same time. This # prevents skipping one of two entries with the same time until the next pass. DELTA = .0001 # We count the number of times a file has been moved to .bak and recovered. # In order to prevent loops and a message flood, when the count reaches this # value, we move the file to the bad queue as a .psv. MAX_BAK_COUNT = 3 elog = logging.getLogger('mailman.error') @public @implementer(ISwitchboard) class Switchboard: """See `ISwitchboard`.""" def __init__(self, name, queue_directory, slice=None, numslices=1, recover=False): """Create a switchboard object. :param name: The queue name. :type name: str :param queue_directory: The queue directory. :type queue_directory: str :param slice: The slice number for this switchboard, or None. If not None, it must be [0..`numslices`). :type slice: int or None :param numslices: The total number of slices to split this queue directory into. It must be a power of 2. :type numslices: int :param recover: True if backup files should be recovered. :type recover: bool """ assert (numslices & (numslices - 1)) == 0, ( 'Not a power of 2: {}'.format(numslices)) self.name = name self.queue_directory = queue_directory # If configured to, create the directory if it doesn't yet exist. if config.create_paths: makedirs(self.queue_directory, 0o770) # Fast track for no slices self._lower = None self._upper = None # BAW: test performance and end-cases of this algorithm if numslices != 1: self._lower = ((shamax + 1) * slice) / numslices self._upper = (((shamax + 1) * (slice + 1)) / numslices) - 1 if recover: self.recover_backup_files() def enqueue(self, _msg, _metadata=None, **_kws): """See `ISwitchboard`.""" if _metadata is None: _metadata = {} # Calculate the SHA hexdigest of the message to get a unique base # filename. We're also going to use the digest as a hash into the set # of parallel runner processes. data = _metadata.copy() data.update(_kws) list_id = data.get('listid', '--nolist--') # Get some data for the input to the sha hash. now = repr(time.time()) if data.get('_plaintext'): protocol = 0 msgsave = pickle.dumps(str(_msg), protocol) else: protocol = pickle.HIGHEST_PROTOCOL msgsave = pickle.dumps(_msg, protocol) # The list-id field is a string but the input to the hash function must # be bytes. hashfood = msgsave + list_id.encode('utf-8') + now.encode('utf-8') # Encode the current time into the file name for FIFO sorting. The # file name consists of two parts separated by a '+': the received # time for this message (i.e. when it first showed up on this system) # and the sha hex digest. filebase = now + '+' + hashlib.sha1(hashfood).hexdigest() filename = os.path.join(self.queue_directory, filebase + '.pck') tmpfile = filename + '.tmp' # Always add the metadata schema version number data['version'] = config.QFILE_SCHEMA_VERSION # Filter out volatile entries. Use .keys() so that we can mutate the # dictionary during the iteration. for k in list(data): if k.startswith('_'): del data[k] # We have to tell the dequeue() method whether to parse the message # object or not. data['_parsemsg'] = (protocol == 0) # Write to the pickle file the message object and metadata. with open(tmpfile, 'wb') as fp: fp.write(msgsave) pickle.dump(data, fp, protocol) fp.flush() os.fsync(fp.fileno()) os.rename(tmpfile, filename) return filebase def dequeue(self, filebase): """See `ISwitchboard`.""" # Calculate the filename from the given filebase. filename = os.path.join(self.queue_directory, filebase + '.pck') backfile = os.path.join(self.queue_directory, filebase + '.bak') # Read the message object and metadata. with open(filename, 'rb') as fp: # Move the file to the backup file name for processing. If this # process crashes uncleanly the .bak file will be used to # re-instate the .pck file in order to try again. os.rename(filename, backfile) msg = pickle.load(fp) data = pickle.load(fp) if data.get('_parsemsg'): # Calculate the original size of the text now so that we won't # have to generate the message later when we do size restriction # checking. original_size = len(msg) msg = email.message_from_string(msg, Message) msg.original_size = original_size data['original_size'] = original_size return msg, data def finish(self, filebase, preserve=False): """See `ISwitchboard`.""" bakfile = os.path.join(self.queue_directory, filebase + '.bak') # It is possible for a queue entry to be created by a non-Mailman user # and not be readable by the Mailman user:group. If this happens, we # get here and the file is a .pck rather than a .bak. pckfile = os.path.join(self.queue_directory, filebase + '.pck') if not os.path.isfile(bakfile) and os.path.isfile(pckfile): # We have a .pck and not a .bak so switch the name for the next. bakfile = pckfile try: if preserve: bad_dir = config.switchboards['bad'].queue_directory psvfile = os.path.join(bad_dir, filebase + '.psv') os.rename(bakfile, psvfile) else: os.unlink(bakfile) except EnvironmentError: elog.exception( 'Failed to unlink/preserve backup file: %s', bakfile) @property def files(self): """See `ISwitchboard`.""" return self.get_files() def get_files(self, extension='.pck'): """See `ISwitchboard`.""" times = {} lower = self._lower upper = self._upper for f in os.listdir(self.queue_directory): # By ignoring anything that doesn't end in .pck, we ignore # tempfiles and avoid a race condition. filebase, ext = os.path.splitext(f) if ext != extension: continue when, digest = filebase.split('+', 1) # Throw out any files which don't match our bitrange. BAW: test # performance and end-cases of this algorithm. MAS: both # comparisons need to be <= to get complete range. if lower is None or (lower <= int(digest, 16) <= upper): key = float(when) while key in times: key += DELTA times[key] = filebase # FIFO sort return [times[k] for k in sorted(times)] def recover_backup_files(self): """See `ISwitchboard`.""" # Move all .bak files in our slice to .pck. It's impossible for both # to exist at the same time, so the move is enough to ensure that our # normal dequeuing process will handle them. We keep count in # _bak_count in the metadata of the number of times we recover this # file. When the count reaches MAX_BAK_COUNT, we move the .bak file # to a .psv file in the bad queue. for filebase in self.get_files('.bak'): src = os.path.join(self.queue_directory, filebase + '.bak') dst = os.path.join(self.queue_directory, filebase + '.pck') with open(src, 'rb+') as fp: try: # Throw away the message object. pickle.load(fp) data_pos = fp.tell() data = pickle.load(fp) except Exception as error: # If unpickling throws any exception, just log and # preserve this entry elog.error('Unpickling .bak exception: %s\n' 'Preserving file: %s', error, filebase) self.finish(filebase, preserve=True) else: data['_bak_count'] = data.get('_bak_count', 0) + 1 fp.seek(data_pos) if data.get('_parsemsg'): protocol = 0 else: protocol = 1 pickle.dump(data, fp, protocol) fp.truncate() fp.flush() os.fsync(fp.fileno()) if data['_bak_count'] >= MAX_BAK_COUNT: elog.error('.bak file max count, preserving file: %s', filebase) self.finish(filebase, preserve=True) else: os.rename(src, dst) @public def handle_ConfigurationUpdatedEvent(event): """Initialize the global switchboards for input/output.""" if not isinstance(event, ConfigurationUpdatedEvent): return config = event.config for conf in config.runner_configs: name = conf.name.split('.')[-1] assert name not in config.switchboards, ( 'Duplicate runner name: {0}'.format(name)) # Path is empty for non-queue runners. Check for path and create # switchboard instances only for queue runners. if conf.path: substitutions = config.paths substitutions['name'] = name path = expand(conf.path, None, substitutions) config.switchboards[name] = Switchboard(name, path) mailman-3.2.2/src/mailman/core/system.py0000644000175000017500000000231013442110351021273 0ustar maxkingmaxking00000000000000# Copyright (C) 2009-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """System information.""" import sys from mailman import version from mailman.interfaces.system import ISystem from public import public from zope.interface import implementer @implementer(ISystem) class System: """See `ISystem`.""" @property def mailman_version(self): """See `ISystem`.""" return version.MAILMAN_VERSION_FULL @property def python_version(self): """See `ISystem`.""" return sys.version public(system=System()) mailman-3.2.2/src/mailman/core/tests/0000755000175000017500000000000013445614541020560 5ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/core/tests/__init__.py0000644000175000017500000000000013244427337022661 0ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/core/tests/test_logging.py0000644000175000017500000000176413442110351023612 0ustar maxkingmaxking00000000000000# Copyright (C) 2012-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Test logging behavior.""" import unittest from mailman.core.logging import ReopenableFileHandler class TestRunner(unittest.TestCase): def test_opening_character_device(self): handler = ReopenableFileHandler('test', '/dev/stdout') handler.close() mailman-3.2.2/src/mailman/core/tests/test_pipelines.py0000644000175000017500000002074613442110351024155 0ustar maxkingmaxking00000000000000# Copyright (C) 2012-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Test the core modification pipelines.""" import unittest from mailman.app.lifecycle import create_list from mailman.config import config from mailman.core.pipelines import process from mailman.interfaces.handler import IHandler from mailman.interfaces.member import MemberRole from mailman.interfaces.pipeline import ( DiscardMessage, IPipeline, RejectMessage) from mailman.interfaces.usermanager import IUserManager from mailman.testing.helpers import ( LogFileMark, digest_mbox, get_queue_messages, specialized_message_from_string as mfs) from mailman.testing.layers import ConfigLayer from operator import delitem from zope.component import getUtility from zope.interface import implementer @implementer(IHandler) class DiscardingHandler: name = 'discarding' def process(self, mlist, msg, msgdata): raise DiscardMessage('by test handler') @implementer(IHandler) class RejectHandler: name = 'rejecting' def __init__(self, message): self.message = message def process(self, mlist, msg, msgdata): raise RejectMessage(self.message) @implementer(IPipeline) class DiscardingPipeline: name = 'test-discarding' description = 'Discarding test pipeline' def __iter__(self): yield DiscardingHandler() @implementer(IPipeline) class RejectingPipeline: name = 'test-rejecting' description = 'Rejectinging test pipeline' def __init__(self): self.message = 'by test handler' def __iter__(self): yield RejectHandler(self.message) class TestPostingPipeline(unittest.TestCase): """Test various aspects of the built-in postings pipeline.""" layer = ConfigLayer def setUp(self): self._mlist = create_list('test@example.com') config.pipelines['test-discarding'] = DiscardingPipeline() self.addCleanup(delitem, config.pipelines, 'test-discarding') config.pipelines['test-rejecting'] = RejectingPipeline() self.addCleanup(delitem, config.pipelines, 'test-rejecting') self._msg = mfs("""\ From: Anne Person To: test@example.com Subject: a test Message-ID: testing """) def test_rfc2369_headers(self): # Ensure that RFC 2369 List-* headers are added. msgdata = {} process(self._mlist, self._msg, msgdata, pipeline_name='default-posting-pipeline') self.assertEqual(self._msg['list-id'], '') self.assertEqual(self._msg['list-post'], '') def test_discarding_pipeline(self): # If a handler in the pipeline raises DiscardMessage, the message will # be thrown away, but with a log message. mark = LogFileMark('mailman.vette') process(self._mlist, self._msg, {}, 'test-discarding') line = mark.readline()[:-1] self.assertTrue(line.endswith( ' discarded by "test-discarding" pipeline handler ' '"discarding": by test handler')) def test_rejecting_pipeline(self): # If a handler in the pipeline raises RejectMessage, the post will # be bounced with a log message. mark = LogFileMark('mailman.vette') process(self._mlist, self._msg, {}, 'test-rejecting') line = mark.readline()[:-1] self.assertEqual( line[-80:], ' rejected by "test-rejecting" pipeline handler ' '"rejecting": by test handler', line) # In the rejection case, the original message will also be in the # virgin queue. items = get_queue_messages('virgin', expected_count=1) self.assertEqual( str(items[0].msg.get_payload(1).get_payload(0)['subject']), 'a test') # The first payload contains the rejection reason. payload = items[0].msg.get_payload(0).get_payload() self.assertEqual(payload, 'by test handler') def test_rejecting_pipeline_without_message(self): # Similar to above, but without a rejection message. pipeline = config.pipelines['test-rejecting'] message = pipeline.message self.addCleanup(setattr, pipeline, 'message', message) pipeline.message = None mark = LogFileMark('mailman.vette') process(self._mlist, self._msg, {}, 'test-rejecting') line = mark.readline()[:-1] self.assertEqual( line[-91:], ' rejected by "test-rejecting" pipeline handler ' '"rejecting": [No details are available]', line) # In the rejection case, the original message will also be in the # virgin queue. items = get_queue_messages('virgin', expected_count=1) self.assertEqual( str(items[0].msg.get_payload(1).get_payload(0)['subject']), 'a test') # The first payload contains the rejection reason. payload = items[0].msg.get_payload(0).get_payload() self.assertEqual(payload, '[No details are available]') def test_decorate_bulk(self): # Ensure that bulk postings get decorated with the footer. # Decorate is no longer in the pipeline. This is now tested in # mailman/mta/tests/test_delivery.py. pass def test_nodecorate_verp(self): # Ensure that verp postings don't get decorated twice. # This test was always wrong as verp decoration wasn't done in # the pipeline. pass def test_only_decorate_output(self): # Ensure that decoration is not done on the archive, digest, or # usenet copy of the message. # This test is moot now that decorate isn't in the posting pipeline # but let it remain. self.assertTrue(self._mlist.digests_enabled) # Set up NNTP. self._mlist.gateway_to_news = True self._mlist.linked_newsgroup = 'testing' self._mlist.nntp_host = 'news.example.com' # Process the email. process(self._mlist, self._msg, {}, pipeline_name='default-posting-pipeline') for queue in ('archive', 'nntp'): items = get_queue_messages(queue, expected_count=1) payload = items[0].msg.get_payload() self.assertNotIn('Test mailing list', payload) self.assertEqual(len(digest_mbox(self._mlist)), 1) payload = digest_mbox(self._mlist)[0].get_payload() self.assertNotIn('Test mailing list', payload) class TestOwnerPipeline(unittest.TestCase): """Test various aspects of the built-in owner pipeline.""" layer = ConfigLayer def setUp(self): self._mlist = create_list('test@example.com') user_manager = getUtility(IUserManager) anne = user_manager.create_address('anne@example.com') bart = user_manager.create_address('bart@example.com') self._mlist.subscribe(anne, MemberRole.owner) self._mlist.subscribe(bart, MemberRole.moderator) self._msg = mfs("""\ From: Anne Person To: test-owner@example.com """) def test_calculate_recipients(self): # Recipients are the administrators of the mailing list. msgdata = dict(listname='test@example.com', to_owner=True) process(self._mlist, self._msg, msgdata, pipeline_name='default-owner-pipeline') self.assertEqual(msgdata['recipients'], set(('anne@example.com', 'bart@example.com'))) def test_to_outgoing(self): # The message, with the calculated recipients, gets put in the # outgoing queue. process(self._mlist, self._msg, {}, pipeline_name='default-owner-pipeline') items = get_queue_messages('out', sort_on='to', expected_count=1) self.assertEqual(items[0].msgdata['recipients'], set(('anne@example.com', 'bart@example.com'))) mailman-3.2.2/src/mailman/core/tests/test_runner.py0000644000175000017500000001211413442110351023464 0ustar maxkingmaxking00000000000000# Copyright (C) 2012-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Test some Runner base class behavior.""" import unittest from mailman.app.lifecycle import create_list from mailman.config import config from mailman.core.runner import Runner from mailman.interfaces.member import DeliveryMode from mailman.interfaces.runner import RunnerCrashEvent from mailman.runners.virgin import VirginRunner from mailman.testing.helpers import ( LogFileMark, configuration, event_subscribers, get_queue_messages, make_digest_messages, make_testable_runner, specialized_message_from_string as mfs, subscribe) from mailman.testing.layers import ConfigLayer class CrashingRunner(Runner): def _dispose(self, mlist, msg, msgdata): raise RuntimeError('borked') class TestRunner(unittest.TestCase): """Test the Runner base class behavior.""" layer = ConfigLayer def setUp(self): self._mlist = create_list('test@example.com') self._events = [] def _got_event(self, event): self._events.append(event) @configuration('runner.crashing', **{'class': 'mailman.core.tests.CrashingRunner'}) def test_crash_event(self): runner = make_testable_runner(CrashingRunner, 'in') # When an exception occurs in Runner._process_one_file(), a zope.event # gets triggered containing the exception object. msg = mfs("""\ From: anne@example.com To: test@example.com Message-ID: """) config.switchboards['in'].enqueue(msg, listid='test.example.com') with event_subscribers(self._got_event): runner.run() # We should now have exactly one event, which will contain the # exception, plus additional metadata containing the mailing list, # message, and metadata. self.assertEqual(len(self._events), 1) event = self._events[0] self.assertIsInstance(event, RunnerCrashEvent) self.assertEqual(event.mailing_list, self._mlist) self.assertEqual(event.message['message-id'], '') self.assertEqual(event.metadata['listid'], 'test.example.com') self.assertIsInstance(event.error, RuntimeError) self.assertEqual(str(event.error), 'borked') self.assertIsInstance(event.runner, CrashingRunner) # The message should also have ended up in the shunt queue. items = get_queue_messages('shunt', expected_count=1) self.assertEqual(items[0].msg['message-id'], '') def test_digest_messages(self): # In LP: #1130697, the digest runner creates MIME digests using the # stdlib MIMEMutlipart class, however this class does not have the # extended attributes we require (e.g. .sender). The fix is to use a # subclass of MIMEMultipart and our own Message subclass; this adds # back the required attributes. (LP: #1130696) self._mlist.send_welcome_message = False # Subscribe some users receiving digests. anne = subscribe(self._mlist, 'Anne') anne.preferences.delivery_mode = DeliveryMode.mime_digests bart = subscribe(self._mlist, 'Bart') bart.preferences.delivery_mode = DeliveryMode.plaintext_digests # Start by creating the raw ingredients for the digests. This also # runs the digest runner, thus producing the digest messages into the # virgin queue. make_digest_messages(self._mlist) # Run the virgin queue processor, which runs the cook-headers and # to-outgoing handlers. This should produce no error. error_log = LogFileMark('mailman.error') runner = make_testable_runner(VirginRunner, 'virgin') runner.run() error_text = error_log.read() self.assertEqual(len(error_text), 0, error_text) get_queue_messages('shunt', expected_count=0) items = get_queue_messages('out', expected_count=2) # Which one is the MIME digest? mime_digest = None for item in items: if item.msg.get_content_type() == 'multipart/mixed': assert mime_digest is None, 'Found two MIME digests' mime_digest = item.msg # The cook-headers handler ran. self.assertIn('x-mailman-version', mime_digest) self.assertEqual(mime_digest['precedence'], 'list') # The list's -request address is the original sender. self.assertEqual(item.msgdata['original_sender'], 'test-request@example.com') mailman-3.2.2/src/mailman/core/tests/test_switchboard.py0000644000175000017500000000510413442110351024465 0ustar maxkingmaxking00000000000000# Copyright (C) 2015-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Switchboard tests.""" import os import unittest from mailman.config import config from mailman.testing.helpers import ( LogFileMark, specialized_message_from_string as mfs) from mailman.testing.layers import ConfigLayer from unittest.mock import patch class TestSwitchboard(unittest.TestCase): layer = ConfigLayer def test_log_exception_in_finish(self): # If something bad happens in .finish(), the traceback should get # logged. LP: #1165589. msg = mfs("""\ From: anne@example.com To: test@example.com Message-ID: """) switchboard = config.switchboards['shunt'] # Enqueue the message. filebase = switchboard.enqueue(msg) error_log = LogFileMark('mailman.error') msg, data = switchboard.dequeue(filebase) # Now, cause .finish() to throw an exception. with patch('mailman.core.switchboard.os.rename', side_effect=OSError('Oops!')): switchboard.finish(filebase, preserve=True) traceback = error_log.read().splitlines() self.assertEqual(traceback[1], 'Traceback (most recent call last):') self.assertEqual(traceback[-1], 'OSError: Oops!') def test_no_bak_but_pck(self): # if there is no .bak file but a .pck with the same filebase, # .finish() should handle the .pck. msg = mfs("""\ From: anne@example.com To: test@example.com Message-ID: """) switchboard = config.switchboards['shunt'] # Enqueue the message. filebase = switchboard.enqueue(msg) # Now call .finish() without first dequeueing. switchboard.finish(filebase, preserve=True) # And ensure the file got preserved. bad_dir = config.switchboards['bad'].queue_directory psvfile = os.path.join(bad_dir, filebase + '.psv') self.assertTrue(os.path.isfile(psvfile)) mailman-3.2.2/src/mailman/database/0000755000175000017500000000000013445614541020232 5ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/database/__init__.py0000644000175000017500000000000013244427337022333 0ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/database/alembic/0000755000175000017500000000000013445614541021626 5ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/database/alembic/__init__.py0000644000175000017500000000205313442110351023722 0ustar maxkingmaxking00000000000000# Copyright (C) 2014-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Alembic configuration initization.""" from alembic.config import Config from contextlib import ExitStack from mailman.utilities.modules import expand_path from public import public with ExitStack() as resources: cfg_path = expand_path(resources, 'python:mailman.config.alembic') public(alembic_cfg=Config(cfg_path)) mailman-3.2.2/src/mailman/database/alembic/env.py0000644000175000017500000000452113442110351022755 0ustar maxkingmaxking00000000000000# Copyright (C) 2014-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Alembic migration environment.""" from alembic import context from contextlib import closing from mailman.config import config from mailman.core.initialize import initialize_1 from mailman.database.model import Model from mailman.utilities.string import expand from sqlalchemy import create_engine try: url = expand(config.database.url, None, config.paths) except AttributeError: # Initialize config object for external alembic calls initialize_1() url = expand(config.database.url, None, config.paths) # We can't use @public here. See GL#423 def run_migrations_offline(): """Run migrations in 'offline' mode. This configures the context with just a URL and not an Engine, though an Engine is acceptable here as well. By skipping the Engine creation we don't even need a DBAPI to be available. Calls to context.execute() here emit the given string to the script output. """ context.configure(url=url, target_metadata=Model.metadata) with context.begin_transaction(): context.run_migrations() # We can't use @public here. See GL#423 def run_migrations_online(): """Run migrations in 'online' mode. In this scenario we need to create an Engine and associate a connection with the context. """ engine = create_engine(url) connection = engine.connect() with closing(connection): context.configure( connection=connection, target_metadata=Model.metadata) with context.begin_transaction(): context.run_migrations() if context.is_offline_mode(): run_migrations_offline() else: run_migrations_online() mailman-3.2.2/src/mailman/database/alembic/script.py.mako0000644000175000017500000000073113244427337024435 0ustar maxkingmaxking00000000000000"""${message} Revision ID: ${up_revision} Revises: ${down_revision} Create Date: ${create_date} """ # revision identifiers, used by Alembic. revision = ${repr(up_revision)} down_revision = ${repr(down_revision)} import sqlalchemy as sa from alembic import op from mailman.database.helpers import is_sqlite, exists_in_db ${imports if imports else ""} def upgrade(): ${upgrades if upgrades else "pass"} def downgrade(): ${downgrades if downgrades else "pass"} mailman-3.2.2/src/mailman/database/alembic/versions/0000755000175000017500000000000013445614541023476 5ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/database/alembic/versions/15401063d4e3_roster_visibility.py0000644000175000017500000000161713432413340031273 0ustar maxkingmaxking00000000000000"""roster_visibility Revision ID: 15401063d4e3 Revises: b2e694dfde35 Create Date: 2019-01-20 20:45:50.773097 """ # revision identifiers, used by Alembic. import sqlalchemy as sa from alembic import op from mailman.database.helpers import exists_in_db, is_sqlite revision = '15401063d4e3' down_revision = 'b2e694dfde35' def upgrade(): # ### commands auto generated by Alembic - please adjust! ### if not exists_in_db( op.get_bind(), 'mailinglist', 'member_roster_visibility'): op.add_column( # pragma: nocover 'mailinglist', sa.Column('member_roster_visibility', sa.Integer(), nullable=True)) def downgrade(): # ### commands auto generated by Alembic - please adjust! ### if not is_sqlite(op.get_bind()): op.drop_column('mailinglist', 'member_roster_visibility') # noqa: E501 # pragma: nocover mailman-3.2.2/src/mailman/database/alembic/versions/16c2b25c7b_list_subscription_policy.py0000644000175000017500000000262313244427337032653 0ustar maxkingmaxking00000000000000"""List subscription policy Revision ID: 16c2b25c7b Revises: 46e92facee7 Create Date: 2015-03-21 11:00:44.634883 """ import sqlalchemy as sa from alembic import op from mailman.database.helpers import exists_in_db, is_sqlite from mailman.database.types import Enum from mailman.interfaces.mailinglist import SubscriptionPolicy # Revision identifiers, used by Alembic. revision = '16c2b25c7b' down_revision = '46e92facee7' def upgrade(): # Update the schema. if not exists_in_db(op.get_bind(), 'mailinglist', 'subscription_policy'): # SQLite may not have removed it when downgrading. op.add_column('mailinglist', sa.Column( 'subscription_policy', Enum(SubscriptionPolicy), nullable=True)) # Now migrate the data. Don't import the table definition from the # models, it may break this migration when the model is updated in the # future (see the Alembic doc). mlist = sa.sql.table( 'mailinglist', sa.sql.column('subscription_policy', Enum(SubscriptionPolicy)) ) # There were no enforced subscription policy before, so all lists are # considered open. op.execute(mlist.update().values( {'subscription_policy': op.inline_literal(SubscriptionPolicy.open)})) def downgrade(): if not is_sqlite(op.get_bind()): # SQLite does not support dropping columns. op.drop_column('mailinglist', 'subscription_policy') mailman-3.2.2/src/mailman/database/alembic/versions/2bb9b382198_workflow_state_table.py0000644000175000017500000000130113244427337031745 0ustar maxkingmaxking00000000000000"""Workflow state table Revision ID: 2bb9b382198 Revises: 16c2b25c7b Create Date: 2015-03-25 18:09:18.338790 """ import sqlalchemy as sa from alembic import op from mailman.database.types import SAUnicode # Revision identifiers, used by Alembic. revision = '2bb9b382198' down_revision = '16c2b25c7b' def upgrade(): op.create_table( 'workflowstate', sa.Column('name', SAUnicode(), nullable=False), sa.Column('token', SAUnicode(), nullable=False), sa.Column('step', SAUnicode(), nullable=True), sa.Column('data', SAUnicode(), nullable=True), sa.PrimaryKeyConstraint('name', 'token') ) def downgrade(): op.drop_table('workflowstate') mailman-3.2.2/src/mailman/database/alembic/versions/3002bac0c25a_dmarc_attributes.py0000644000175000017500000000472613244427337031264 0ustar maxkingmaxking00000000000000"""dmarc_attributes Revision ID: 3002bac0c25a Revises: a46993b05703 Create Date: 2016-10-30 22:05:17.881880 """ import sqlalchemy as sa from alembic import op from mailman.database.helpers import exists_in_db from mailman.database.types import Enum, SAUnicodeLarge from mailman.interfaces.mailinglist import DMARCMitigateAction # revision identifiers, used by Alembic. revision = '3002bac0c25a' down_revision = 'a46993b05703' def upgrade(): if not exists_in_db(op.get_bind(), 'mailinglist', 'dmarc_mitigate_action' ): # SQLite may not have removed it when downgrading. It should be OK # to just test one. op.add_column('mailinglist', sa.Column( 'dmarc_mitigate_action', Enum(DMARCMitigateAction), nullable=True)) op.add_column('mailinglist', sa.Column( 'dmarc_mitigate_unconditionally', sa.Boolean, nullable=True)) op.add_column('mailinglist', sa.Column( 'dmarc_moderation_notice', SAUnicodeLarge(), nullable=True)) op.add_column('mailinglist', sa.Column( 'dmarc_wrapped_message_text', SAUnicodeLarge(), nullable=True)) # Now migrate the data. Don't import the table definition from the # models, it may break this migration when the model is updated in the # future (see the Alembic doc). mlist = sa.sql.table( 'mailinglist', sa.sql.column('dmarc_mitigate_action', Enum(DMARCMitigateAction)), sa.sql.column('dmarc_mitigate_unconditionally', sa.Boolean), sa.sql.column('dmarc_moderation_notice', SAUnicodeLarge()), sa.sql.column('dmarc_wrapped_message_text', SAUnicodeLarge()), ) # These are all new attributes so just set defaults. op.execute(mlist.update().values(dict( dmarc_mitigate_action=op.inline_literal( DMARCMitigateAction.no_mitigation), dmarc_mitigate_unconditionally=op.inline_literal(False), dmarc_moderation_notice=op.inline_literal(''), dmarc_wrapped_message_text=op.inline_literal(''), ))) def downgrade(): with op.batch_alter_table('mailinglist') as batch_op: batch_op.drop_column('dmarc_mitigate_action') batch_op.drop_column('dmarc_mitigate_unconditionally') batch_op.drop_column('dmarc_moderation_notice') batch_op.drop_column('dmarc_wrapped_message_text') mailman-3.2.2/src/mailman/database/alembic/versions/33bc0099223_add_member_indexes.py0000644000175000017500000000271613244427337031241 0ustar maxkingmaxking00000000000000"""Add member indexes Revision ID: 33bc0099223 Revises: 42756496720 Create Date: 2015-11-19 23:04:42.449553 """ from alembic import op from mailman.database.helpers import is_mysql # Revision identifiers, used by Alembic. revision = '33bc0099223' down_revision = '42756496720' def upgrade(): op.create_index(op.f('ix_address_email'), 'address', ['email'], unique=False) # MySQL automatically creates the indexes for primary keys so don't need # to do it explicitly again. if not is_mysql(op.get_bind()): op.create_index(op.f('ix_member_address_id'), 'member', ['address_id'], unique=False) op.create_index(op.f('ix_member_preferences_id'), 'member', ['preferences_id'], unique=False) op.create_index(op.f('ix_member_user_id'), 'member', ['user_id'], unique=False) def downgrade(): op.drop_index(op.f('ix_address_email'), table_name='address') # MySQL automatically creates and removes the indexes for primary keys. # So, you cannot drop it without removing the foreign key constraint. if not is_mysql(op.get_bind()): op.drop_index(op.f('ix_member_user_id'), table_name='member') op.drop_index(op.f('ix_member_preferences_id'), table_name='member') op.drop_index(op.f('ix_member_address_id'), table_name='member') mailman-3.2.2/src/mailman/database/alembic/versions/33e1f5f6fa8_.py0000644000175000017500000000417113442110351025736 0ustar maxkingmaxking00000000000000# Copyright (C) 2015-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Migration from Python 2 to Python 3. Some columns changed from LargeBinary type to Unicode type. Revision ID: 33e1f5f6fa8 Revises: 51b7f92bd06c Create Date: 2015-01-20 17:32:30.144083 """ import sqlalchemy as sa from alembic import op from mailman.database.helpers import is_sqlite from mailman.database.types import SAUnicode # Revision identifiers, used by Alembic. revision = '33e1f5f6fa8' down_revision = '51b7f92bd06c' COLUMNS_TO_CHANGE = ( ('message', 'message_id_hash'), ('message', 'path'), ('pended', 'token'), ('_request', 'data_hash'), ('user', 'password'), ) def upgrade(): if is_sqlite(op.get_bind()): # SQLite does not support altering columns. return for table, column in COLUMNS_TO_CHANGE: op.alter_column(table, column, type_=SAUnicode) def downgrade(): if is_sqlite(op.get_bind()): # SQLite does not support altering columns. return for table, column in COLUMNS_TO_CHANGE: if op.get_bind().dialect.name == 'postgresql': # PostgreSQL needs the USING clause that Alembic does not support # yet. op.execute( ('ALTER TABLE "{table}" ALTER COLUMN "{column}" ' 'TYPE BYTEA USING decode("{column}", \'UTF8\')').format( table=table, column=column)) else: op.alter_column(table, column, type_=sa.LargeBinary) mailman-3.2.2/src/mailman/database/alembic/versions/3e09bb4a5dc_member_indexes.py0000644000175000017500000000112313244427337030722 0ustar maxkingmaxking00000000000000"""Add indexes on the Member table. Revision ID: 3e09bb4a5dc Revises: 33bc0099223 Create Date: 2015-12-11 19:16:57.030395 """ from alembic import op # Revision identifiers, used by Alembic. revision = '3e09bb4a5dc' down_revision = '33bc0099223' def upgrade(): op.create_index( op.f('ix_member_list_id'), 'member', ['list_id'], unique=False) op.create_index( op.f('ix_member_role'), 'member', ['role'], unique=False) def downgrade(): op.drop_index(op.f('ix_member_role'), table_name='member') op.drop_index(op.f('ix_member_list_id'), table_name='member') ././@LongLink0000000000000000000000000000015300000000000011214 Lustar 00000000000000mailman-3.2.2/src/mailman/database/alembic/versions/3f31035ed0d7_further_increase_pendedkeyvalue_value_.pymailman-3.2.2/src/mailman/database/alembic/versions/3f31035ed0d7_further_increase_pendedkeyvalue_val0000644000175000017500000000247013421245331034502 0ustar maxkingmaxking00000000000000"""further_increase_pendedkeyvalue_value_size Revision ID: 3f31035ed0d7 Revises: 4bd95c99b2e7 Create Date: 2017-10-18 17:42:35.550686 """ from alembic import op from mailman.database.types import SAUnicodeLarge, SAUnicodeXL # revision identifiers, used by Alembic. revision = '3f31035ed0d7' down_revision = '4bd95c99b2e7' def upgrade(): # pendedkeyvalue table values can be much larger than SAUnicodeLarge with op.batch_alter_table('pendedkeyvalue') as batch_op: # Drop the existing index on the table. batch_op.drop_index(op.f('ix_pendedkeyvalue_value')) # Alter the column type and then create a new index with # mysql_length set to a fixed length value. batch_op.alter_column('value', type_=SAUnicodeXL) batch_op.create_index(op.f('ix_pendedkeyvalue_value'), columns=['value'], mysql_length=100) def downgrade(): with op.batch_alter_table('pendedkeyvalue') as batch_op: batch_op.alter_column('value', type_=SAUnicodeLarge) # Drop the existing index because it has a fixed length value and then # re-create without the length constraint. batch_op.drop_index(op.f('ix_pendedkeyvalue_value')) batch_op.create_index(op.f('ix_pendedkeyvalue_value'), columns=['value']) mailman-3.2.2/src/mailman/database/alembic/versions/42756496720_header_matches.py0000644000175000017500000000652613244427337030262 0ustar maxkingmaxking00000000000000"""header_matches Revision ID: 42756496720 Revises: 2bb9b382198 Create Date: 2015-09-11 10:11:38.310315 """ import sqlalchemy as sa from alembic import op from mailman.database.helpers import exists_in_db, is_sqlite from mailman.database.types import SAUnicode # Revision identifiers, used by Alembic. revision = '42756496720' down_revision = '2bb9b382198' def upgrade(): # Create the new table header_match_table = op.create_table( 'headermatch', sa.Column('id', sa.Integer(), nullable=False), sa.Column('mailing_list_id', sa.Integer(), nullable=True), sa.Column('header', SAUnicode(), nullable=False), sa.Column('pattern', SAUnicode(), nullable=False), sa.Column('chain', SAUnicode(), nullable=True), sa.ForeignKeyConstraint(['mailing_list_id'], ['mailinglist.id'], ), sa.PrimaryKeyConstraint('id') ) # Now migrate the data. It can't be offline because we need to read the # pickles. connection = op.get_bind() # Don't import the table definition from the models, it may break this # migration when the model is updated in the future (see the Alembic doc). mlist_table = sa.sql.table( 'mailinglist', sa.sql.column('id', sa.Integer), sa.sql.column('header_matches', sa.PickleType) ) for mlist_id, old_matches in connection.execute(mlist_table.select()): for old_match in old_matches: connection.execute(header_match_table.insert().values( mailing_list_id=mlist_id, header=old_match[0], pattern=old_match[1], chain=None )) # Now that data is migrated, drop the old column (except on SQLite which # does not support this) if not is_sqlite(connection): op.drop_column('mailinglist', 'header_matches') def downgrade(): if not exists_in_db(op.get_bind(), 'mailinglist', 'header_matches'): # SQLite will not have deleted the former column, since it does not # support column deletion. op.add_column( 'mailinglist', sa.Column('header_matches', sa.PickleType, nullable=True)) # Now migrate the data. It can't be offline because we need to read the # pickles. connection = op.get_bind() # Don't import the table definition from the models, it may break this # migration when the model is updated in the future (see the Alembic doc). mlist_table = sa.sql.table( 'mailinglist', sa.sql.column('id', sa.Integer), sa.sql.column('header_matches', sa.PickleType) ) header_match_table = sa.sql.table( 'headermatch', sa.sql.column('mailing_list_id', sa.Integer), sa.sql.column('header', SAUnicode), sa.sql.column('pattern', SAUnicode), ) for mlist_id, header, pattern in connection.execute( header_match_table.select()).fetchall(): mlist = connection.execute(mlist_table.select().where( mlist_table.c.id == mlist_id)).fetchone() header_matches = mlist['header_matches'] if not header_matches: header_matches = [] header_matches.append((header, pattern)) connection.execute(mlist_table.update().where( mlist_table.c.id == mlist_id).values( header_matches=header_matches)) op.drop_table('headermatch') mailman-3.2.2/src/mailman/database/alembic/versions/448a93984c35_unsubscription_workflow.py0000644000175000017500000000301413244427337032565 0ustar maxkingmaxking00000000000000"""unsubscription_workflow Revision ID: 448a93984c35 Revises: fa0d96e28631 Create Date: 2016-06-02 14:34:24.154723 """ import sqlalchemy as sa from alembic import op from mailman.database.helpers import exists_in_db from mailman.database.types import Enum, SAUnicode from mailman.interfaces.mailinglist import SubscriptionPolicy # revision identifiers, used by Alembic. revision = '448a93984c35' down_revision = 'fa0d96e28631' def upgrade(): if not exists_in_db(op.get_bind(), 'mailinglist', 'unsubscription_policy'): # SQLite may not have removed it when downgrading. op.add_column('mailinglist', sa.Column( 'unsubscription_policy', Enum(SubscriptionPolicy), nullable=True)) # Now migrate the data. Don't import the table definition from the # models, it may break this migration when the model is updated in the # future (see the Alembic doc). mlist = sa.sql.table( 'mailinglist', sa.sql.column('unsubscription_policy', Enum(SubscriptionPolicy)) ) # There was no previous unsubscription policy. op.execute(mlist.update().values( {'unsubscription_policy': op.inline_literal(SubscriptionPolicy.confirm)})) with op.batch_alter_table('workflowstate') as batch_op: batch_op.drop_column('name') def downgrade(): with op.batch_alter_table('mailinglist') as batch_op: batch_op.drop_column('unsubscription_policy') op.add_column('workflowstate', sa.Column('name', SAUnicode)) mailman-3.2.2/src/mailman/database/alembic/versions/46e92facee7_add_serverowner_domainowner.py0000644000175000017500000000422013442110351033530 0ustar maxkingmaxking00000000000000# Copyright (C) 2015-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """add_serverowner_domainowner Revision ID: 46e92facee7 Revises: 33e1f5f6fa8 Create Date: 2015-03-20 16:01:25.007242 """ import sqlalchemy as sa from alembic import op from mailman.database.helpers import exists_in_db, is_sqlite # Revision identifiers, used by Alembic. revision = '46e92facee7' down_revision = '33e1f5f6fa8' def upgrade(): op.create_table( 'domain_owner', sa.Column('user_id', sa.Integer(), nullable=False), sa.Column('domain_id', sa.Integer(), nullable=False), sa.ForeignKeyConstraint(['domain_id'], ['domain.id'], ), sa.ForeignKeyConstraint(['user_id'], ['user.id'], ), sa.PrimaryKeyConstraint('user_id', 'domain_id') ) if not exists_in_db(op.get_bind(), 'user', 'is_server_owner'): # SQLite may not have removed it when downgrading. op.add_column( 'user', sa.Column('is_server_owner', sa.Boolean(), nullable=True)) if not is_sqlite(op.get_bind()): op.drop_column('domain', 'contact_address') def downgrade(): if not is_sqlite(op.get_bind()): op.drop_column('user', 'is_server_owner') if not exists_in_db(op.get_bind(), 'domain', 'contact_address'): # SQLite may not have removed it. Add a fixed length VARCHAR for # MySQL. op.add_column( 'domain', sa.Column('contact_address', sa.VARCHAR(255), nullable=True)) op.drop_table('domain_owner') mailman-3.2.2/src/mailman/database/alembic/versions/47294d3a604_pendable_indexes.py0000644000175000017500000000677513244427337030755 0ustar maxkingmaxking00000000000000"""Pendable indexes Add indexes on Pendable fields that can be queried upon. Revision ID: 47294d3a604 Revises: 33bc0099223 Create Date: 2015-12-02 11:46:47.295174 """ import json import sqlalchemy as sa from alembic import op from mailman.database.types import SAUnicode # revision identifiers, used by Alembic. revision = '47294d3a604' down_revision = '3e09bb4a5dc' TYPE_CLUES = { 'member_id': 'probe', 'token_owner': 'subscription', '_mod_message_id': 'data', } pended_table = sa.sql.table( 'pended', sa.sql.column('id', sa.Integer), ) keyvalue_table = sa.sql.table( 'pendedkeyvalue', sa.sql.column('id', sa.Integer), sa.sql.column('key', SAUnicode), sa.sql.column('value', SAUnicode), sa.sql.column('pended_id', sa.Integer), ) def upgrade(): op.create_index( op.f('ix_pended_expiration_date'), 'pended', ['expiration_date'], unique=False) op.create_index(op.f('ix_pended_token'), 'pended', ['token'], unique=False) op.create_index( op.f('ix_pendedkeyvalue_key'), 'pendedkeyvalue', ['key'], unique=False) op.create_index( op.f('ix_pendedkeyvalue_value'), 'pendedkeyvalue', ['value'], unique=False) # Data migration. connection = op.get_bind() for pended_result in connection.execute(pended_table.select()).fetchall(): pended_id = pended_result['id'] keyvalues = connection.execute(keyvalue_table.select().where( keyvalue_table.c.pended_id == pended_id )).fetchall() kv_type = [kv for kv in keyvalues if kv['key'] == 'type'] if kv_type: # Convert existing type keys from JSON to plain text. # The (pended_id, key) tuple is unique. kv_type = kv_type[0] try: new_value = json.loads(kv_type['value']) except ValueError: # New-style entry (or already converted). pass else: connection.execute(keyvalue_table.update().where( keyvalue_table.c.id == kv_type['id'] ).values(value=new_value)) else: # Detect the type and add the corresponding type key. keys = [kv['key'] for kv in keyvalues] for clue_key, clue_type in TYPE_CLUES.items(): if clue_key not in keys: continue # We found the type, update the DB. connection.execute(keyvalue_table.insert().values( key='type', value=clue_type, pended_id=pended_id)) break def downgrade(): # Data migration. connection = op.get_bind() # Remove the introduced type keys. connection.execute(keyvalue_table.delete().where(sa.and_( keyvalue_table.c.key == 'type', keyvalue_table.c.value.in_(TYPE_CLUES.values()) ))) # Convert the other type keys to JSON. keyvalues = connection.execute(keyvalue_table.select().where( keyvalue_table.c.key == 'type')).fetchall() for keyvalue in keyvalues: connection.execute(keyvalue_table.update().where( keyvalue_table.c.id == keyvalue['id'] ).values(value=json.dumps(keyvalue['value']))) # Remove indexes. op.drop_index(op.f('ix_pendedkeyvalue_value'), table_name='pendedkeyvalue') op.drop_index(op.f('ix_pendedkeyvalue_key'), table_name='pendedkeyvalue') op.drop_index(op.f('ix_pended_token'), table_name='pended') op.drop_index(op.f('ix_pended_expiration_date'), table_name='pended') mailman-3.2.2/src/mailman/database/alembic/versions/4bd95c99b2e7_fix_template_password.py0000644000175000017500000000101313421245331032351 0ustar maxkingmaxking00000000000000"""Fix template password field. Revision ID: 4bd95c99b2e7 Revises: 3002bac0c25a Create Date: 2017-05-24 10:56:41.256602 """ from alembic import op from mailman.database.types import SAUnicode # revision identifiers, used by Alembic. revision = '4bd95c99b2e7' down_revision = '3002bac0c25a' def upgrade(): with op.batch_alter_table('template') as batch_op: batch_op.alter_column('password', type_=SAUnicode) def downgrade(): # Don't go back to DateTime, it will fail if a password was set. pass mailman-3.2.2/src/mailman/database/alembic/versions/51b7f92bd06c_initial.py0000644000175000017500000000404013442110351027360 0ustar maxkingmaxking00000000000000# Copyright (C) 2014-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Initial migration. This empty migration file makes sure there is always an alembic_version in the database. As a consequence, if the database version is reported as None, it means the database needs to be created from scratch with SQLAlchemy itself. It also removes schema items left over from Storm. Revision ID: 51b7f92bd06c Revises: None Create Date: 2014-10-10 09:53:35.624472 """ import sqlalchemy as sa from alembic import op from mailman.database.helpers import exists_in_db, is_sqlite # Revision identifiers, used by Alembic. revision = '51b7f92bd06c' down_revision = None def upgrade(): op.drop_table('version') if not is_sqlite(op.get_bind()): # SQLite does not support dropping columns. op.drop_column('mailinglist', 'acceptable_aliases_id') op.create_index(op.f('ix_user__user_id'), 'user', ['_user_id'], unique=False) op.drop_index('ix_user_user_id', table_name='user') def downgrade(): op.create_table('version') op.create_index('ix_user_user_id', 'user', ['_user_id'], unique=False) op.drop_index(op.f('ix_user__user_id'), table_name='user') if not exists_in_db(op.get_bind(), 'mailinglist', 'acceptable_aliases_id'): op.add_column( 'mailinglist', sa.Column('acceptable_aliases_id', sa.INTEGER(), nullable=True)) mailman-3.2.2/src/mailman/database/alembic/versions/70af5a4e5790_digests.py0000644000175000017500000000345313244427337027342 0ustar maxkingmaxking00000000000000"""digests Revision ID: 70af5a4e5790 Revises: 47294d3a604 Create Date: 2015-12-19 12:05:42.202998 """ import os import sqlalchemy as sa from alembic import op from mailman.config import config # Revision identifiers, used by Alembic. revision = '70af5a4e5790' down_revision = '47294d3a604' def upgrade(): with op.batch_alter_table('mailinglist') as batch_op: batch_op.alter_column('digestable', new_column_name='digests_enabled', existing_type=sa.Boolean) # All column modifications require existing types for Mysql. batch_op.drop_column('nondigestable') # Non-database migration: rename the list's data-path. for dirname in os.listdir(config.LIST_DATA_DIR): if '@' in dirname: old_name = os.path.join(config.LIST_DATA_DIR, dirname) listname, at, domain = dirname.partition('@') new_name = os.path.join(config.LIST_DATA_DIR, '{}.{}'.format(listname, domain)) os.rename(old_name, new_name) def downgrade(): with op.batch_alter_table('mailinglist') as batch_op: batch_op.alter_column('digests_enabled', new_column_name='digestable', existing_type=sa.Boolean) # The data for this column is lost, it's not used anyway. batch_op.add_column(sa.Column('nondigestable', sa.Boolean)) for dirname in os.listdir(config.LIST_DATA_DIR): if '@' not in dirname: old_name = os.path.join(config.LIST_DATA_DIR, dirname) listname, domain = dirname.split('.', 1) new_name = os.path.join(config.LIST_DATA_DIR, '{}@{}'.format(listname, domain)) os.rename(old_name, new_name) mailman-3.2.2/src/mailman/database/alembic/versions/781a38e146bf_mailinglist_indexes.py0000644000175000017500000000152113244427337031727 0ustar maxkingmaxking00000000000000"""MailingList indexes Revision ID: 781a38e146bf Revises: 70af5a4e5790 Create Date: 2016-01-14 15:34:29.734429 """ from alembic import op # Revision identifiers, used by Alembic. revision = '781a38e146bf' down_revision = '70af5a4e5790' def upgrade(): op.create_index( op.f('ix_mailinglist_list_id'), 'mailinglist', ['list_id'], unique=True) op.create_index( op.f('ix_mailinglist_list_name'), 'mailinglist', ['list_name'], unique=False) op.create_index( op.f('ix_mailinglist_mail_host'), 'mailinglist', ['mail_host'], unique=False) def downgrade(): op.drop_index(op.f('ix_mailinglist_mail_host'), table_name='mailinglist') op.drop_index(op.f('ix_mailinglist_list_name'), table_name='mailinglist') op.drop_index(op.f('ix_mailinglist_list_id'), table_name='mailinglist') ././@LongLink0000000000000000000000000000014700000000000011217 Lustar 00000000000000mailman-3.2.2/src/mailman/database/alembic/versions/7b254d88f122_members_and_list_moderation_action.pymailman-3.2.2/src/mailman/database/alembic/versions/7b254d88f122_members_and_list_moderation_action.0000644000175000017500000000600513244427337034331 0ustar maxkingmaxking00000000000000"""Members and list moderation action. Revision ID: 7b254d88f122 Revises: d4fbb4fd34ca Create Date: 2016-02-10 11:31:04.233619 This is a data-only migration. If a member has the same moderation action as the mailing list's default, then set its moderation action to None and use the fallback to the list's default. """ import sqlalchemy as sa from alembic import op from mailman.database.types import Enum, SAUnicode from mailman.interfaces.action import Action from mailman.interfaces.member import MemberRole # Revision identifiers, used by Alembic. revision = '7b254d88f122' down_revision = 'd4fbb4fd34ca' mailinglist_table = sa.sql.table( 'mailinglist', sa.sql.column('id', sa.Integer), sa.sql.column('list_id', SAUnicode), sa.sql.column('default_member_action', Enum(Action)), sa.sql.column('default_nonmember_action', Enum(Action)), ) member_table = sa.sql.table( 'member', sa.sql.column('id', sa.Integer), sa.sql.column('list_id', SAUnicode), sa.sql.column('role', Enum(MemberRole)), sa.sql.column('moderation_action', Enum(Action)), ) # This migration only considers members and nonmembers. members_query = member_table.select().where(sa.or_( member_table.c.role == MemberRole.member, member_table.c.role == MemberRole.nonmember, )) # list-id -> {property-name -> action} # # where property-name will be either default_member_action or # default_nonmember_action. DEFAULT_ACTION_CACHE = {} MISSING = object() def _get_default_action(connection, member): list_id = member['list_id'] property_name = 'default_{}_action'.format(member['role'].name) list_mapping = DEFAULT_ACTION_CACHE.setdefault(list_id, {}) action = list_mapping.get(property_name, MISSING) if action is MISSING: mailing_list = connection.execute(mailinglist_table.select().where( mailinglist_table.c.list_id == list_id)).fetchone() action = mailing_list[property_name] list_mapping[property_name] = action return action def upgrade(): connection = op.get_bind() for member in connection.execute(members_query).fetchall(): default_action = _get_default_action(connection, member) # If the (non)member's moderation action is the same as the mailing # list's default, then set it to None. The moderation rule will # fallback to the list's default. if member['moderation_action'] == default_action: connection.execute(member_table.update().where( member_table.c.id == member['id'] ).values(moderation_action=None)) def downgrade(): connection = op.get_bind() for member in connection.execute(members_query.where( member_table.c.moderation_action == None)).fetchall(): # noqa: E711 default_action = _get_default_action(connection, member) # Use the mailing list's default action connection.execute(member_table.update().where( member_table.c.id == member['id'] ).values(moderation_action=default_action)) mailman-3.2.2/src/mailman/database/alembic/versions/89913992dd5a_remove_mime_is_default_digest.py0000644000175000017500000000151413421245331033661 0ustar maxkingmaxking00000000000000"""remove mime_is_default_digest Revision ID: 89913992dd5a Revises: 448a93984c35 Create Date: 2016-10-31 09:21:24.941438 """ import sqlalchemy as sa from alembic import op from mailman.database.helpers import exists_in_db, is_sqlite # revision identifiers, used by Alembic. revision = '89913992dd5a' down_revision = 'dfe82cf73702' def upgrade(): if not is_sqlite(op.get_bind()): # SQLite does not support dropping columns. op.drop_column( # pragma: nocover 'mailinglist', 'mime_is_default_digest') # pragma: nocover def downgrade(): if not exists_in_db( op.get_bind(), 'mailinglist', 'mime_is_default_digest'): op.add_column( 'mailinglist', sa.Column('mime_is_default_digest', sa.BOOLEAN(), nullable=True)) ././@LongLink0000000000000000000000000000014700000000000011217 Lustar 00000000000000mailman-3.2.2/src/mailman/database/alembic/versions/a46993b05703_increase_pendedkeyvalue_value_size.pymailman-3.2.2/src/mailman/database/alembic/versions/a46993b05703_increase_pendedkeyvalue_value_size.0000644000175000017500000000125413244427337034267 0ustar maxkingmaxking00000000000000"""increase pendedkeyvalue value size Revision ID: a46993b05703 Revises: 448a93984c35 Create Date: 2016-12-15 20:43:48.520837 """ from alembic import op from mailman.database.types import SAUnicode, SAUnicodeLarge # revision identifiers, used by Alembic. revision = 'a46993b05703' down_revision = '448a93984c35' def upgrade(): # Adding another rule can make the rule Hits/Misses too long for MySQL # SaUnicode. with op.batch_alter_table('pendedkeyvalue') as batch_op: batch_op.alter_column('value', type_=SAUnicodeLarge) def downgrade(): with op.batch_alter_table('pendedkeyvalue') as batch_op: batch_op.alter_column('value', type_=SAUnicode) mailman-3.2.2/src/mailman/database/alembic/versions/b2e694dfde35_unique_email_contraint.py0000644000175000017500000000140513421245331032563 0ustar maxkingmaxking00000000000000"""unique email contraint Revision ID: b2e694dfde35 Revises: 89913992dd5a Create Date: 2018-05-17 09:14:01.087221 """ from alembic import op # revision identifiers, used by Alembic. revision = 'b2e694dfde35' down_revision = '89913992dd5a' def upgrade(): # ### commands auto generated by Alembic - please adjust! ### op.drop_index('ix_address_email', table_name='address') op.create_index( op.f('ix_address_email'), 'address', ['email'], unique=True) # ### end Alembic commands ### def downgrade(): # ### commands auto generated by Alembic - please adjust! ### op.drop_index(op.f('ix_address_email'), table_name='address') op.create_index('ix_address_email', 'address', ['email'], unique=False) # ### end Alembic commands ### mailman-3.2.2/src/mailman/database/alembic/versions/bfda02ab3a9b_ban_indexes.py0000644000175000017500000000103313244427337030426 0ustar maxkingmaxking00000000000000"""Ban indexes Revision ID: bfda02ab3a9b Revises: 70af5a4e5790 Create Date: 2016-01-14 16:15:44.059688 """ from alembic import op # Revision identifiers, used by Alembic. revision = 'bfda02ab3a9b' down_revision = '781a38e146bf' def upgrade(): op.create_index(op.f('ix_ban_email'), 'ban', ['email'], unique=False) op.create_index(op.f('ix_ban_list_id'), 'ban', ['list_id'], unique=False) def downgrade(): op.drop_index(op.f('ix_ban_list_id'), table_name='ban') op.drop_index(op.f('ix_ban_email'), table_name='ban') mailman-3.2.2/src/mailman/database/alembic/versions/d4fbb4fd34ca_header_match_order.py0000644000175000017500000000406613244427337031771 0ustar maxkingmaxking00000000000000"""Add a numerical position column to sort header matches. Revision ID: d4fbb4fd34ca Revises: bfda02ab3a9b Create Date: 2016-02-01 15:57:09.807678 """ import sqlalchemy as sa from alembic import op from mailman.database.helpers import is_mysql # Revision identifiers, used by Alembic. revision = 'd4fbb4fd34ca' down_revision = 'bfda02ab3a9b' def upgrade(): with op.batch_alter_table('headermatch') as batch_op: batch_op.add_column( sa.Column('position', sa.Integer(), nullable=True)) batch_op.create_index( op.f('ix_headermatch_position'), ['position'], unique=False) if not is_mysql(op.get_bind()): # MySQL automatically creates indexes for primary keys. batch_op.create_index( op.f('ix_headermatch_mailing_list_id'), ['mailing_list_id'], unique=False) # MySQL doesn't allow changing columns used in a foreign key # constrains since MySQL version 5.6. We need to drop the # constraint before changing the column. But, since the # constraint name is auto-generated, we can't really hardcode the # name here to use batch_op.drop_constraint(). Until we have a # better fix for this, it should be safe to skip this. batch_op.alter_column( 'mailing_list_id', existing_type=sa.INTEGER(), nullable=False) def downgrade(): with op.batch_alter_table('headermatch') as batch_op: batch_op.drop_index(op.f('ix_headermatch_position')) batch_op.drop_column('position') if not is_mysql(op.get_bind()): # MySQL automatically creates and removes the indexes for primary # keys. So, you cannot drop it without removing the foreign key # constraint. batch_op.drop_index(op.f('ix_headermatch_mailing_list_id')) # MySQL doesn't allow changing columns used in foreign_key # constraints. batch_op.alter_column( 'mailing_list_id', existing_type=sa.INTEGER(), nullable=True) mailman-3.2.2/src/mailman/database/alembic/versions/dfe82cf73702_add_alias_domain.py0000644000175000017500000000123513421245331031174 0ustar maxkingmaxking00000000000000"""add_alias_domain Revision ID: dfe82cf73702 Revises: fa0d96e28631 Create Date: 2016-10-07 16:50:53.368932 """ import sqlalchemy as sa from alembic import op from mailman.database.helpers import exists_in_db from mailman.database.types import SAUnicode # revision identifiers, used by Alembic. revision = 'dfe82cf73702' down_revision = '3f31035ed0d7' def upgrade(): if not exists_in_db(op.get_bind(), 'domain', 'alias_domain'): op.add_column( 'domain', sa.Column('alias_domain', SAUnicode, nullable=True) ) def downgrade(): with op.batch_alter_table('domain') as batch_op: batch_op.drop_column('alias_domain') mailman-3.2.2/src/mailman/database/alembic/versions/fa0d96e28631_template_manager.py0000644000175000017500000001540613421245331031174 0ustar maxkingmaxking00000000000000"""File cache and template manager. Revision ID: fa0d96e28631 Revises: 7b254d88f122 Create Date: 2016-02-21 16:21:48.277654 """ import os import shutil import sqlalchemy as sa from alembic import op from mailman.config import config from mailman.database.helpers import exists_in_db from mailman.database.types import SAUnicode # revision identifiers, used by Alembic. revision = 'fa0d96e28631' down_revision = '7b254d88f122' CONVERSION_MAPPING = dict( digest_footer_uri='list:digest:footer', digest_header_uri='list:digest:header', footer_uri='list:regular:footer', goodbye_message_uri='list:user:notice:goodbye', header_uri='list:regular:header', welcome_message_uri='list:user:notice:welcome', ) REVERSE_MAPPING = {value: key for key, value in CONVERSION_MAPPING.items()} def upgrade(): op.create_table( 'file_cache', sa.Column('id', sa.Integer(), nullable=False), sa.Column('key', SAUnicode(), nullable=False), sa.Column('file_id', SAUnicode(), nullable=True), sa.Column('is_bytes', sa.Boolean(), nullable=False), sa.Column('created_on', sa.DateTime(), nullable=False), sa.Column('expires_on', sa.DateTime(), nullable=False), sa.PrimaryKeyConstraint('id') ) template_table = op.create_table( 'template', sa.Column('id', sa.Integer(), nullable=False), sa.Column('name', SAUnicode(), nullable=False), sa.Column('context', SAUnicode(), nullable=True), sa.Column('uri', SAUnicode(), nullable=False), sa.Column('username', SAUnicode(), nullable=True), sa.Column('password', sa.DateTime(), nullable=True), sa.PrimaryKeyConstraint('id') ) connection = op.get_bind() # For all existing mailing lists, turn the *_uri attributes into entries # in the template cache. Don't import the table definition from the # models, it may break this migration when the model is updated in the # future (see the Alembic doc). mlist_table = sa.sql.table( 'mailinglist', sa.sql.column('id', sa.Integer), sa.sql.column('list_id', SAUnicode), sa.sql.column('digest_footer_uri', SAUnicode), sa.sql.column('digest_header_uri', SAUnicode), sa.sql.column('footer_uri', SAUnicode), sa.sql.column('header_uri', SAUnicode), sa.sql.column('goodbye_message_uri', SAUnicode), sa.sql.column('welcome_message_uri', SAUnicode), ) for (mlist_id, list_id, digest_footer_uri, digest_header_uri, nondigest_footer_uri, nondigest_header_uri, goodbye_uri, welcome_uri ) in connection.execute(mlist_table.select()): inserts = [] if digest_footer_uri is not None: entry = dict( name=CONVERSION_MAPPING['digest_footer_uri'], uri=digest_footer_uri, ) inserts.append(entry) if digest_header_uri is not None: entry = dict( name=CONVERSION_MAPPING['digest_header_uri'], uri=digest_header_uri, ) inserts.append(entry) if nondigest_footer_uri is not None: entry = dict( name=CONVERSION_MAPPING['footer_uri'], uri=nondigest_footer_uri, ) inserts.append(entry) if nondigest_header_uri is not None: entry = dict( name=CONVERSION_MAPPING['header_uri'], uri=nondigest_header_uri, ) inserts.append(entry) if goodbye_uri is not None: entry = dict( name=CONVERSION_MAPPING['goodbye_message_uri'], uri=goodbye_uri, ) inserts.append(entry) if welcome_uri is not None: entry = dict( name=CONVERSION_MAPPING['welcome_message_uri'], uri=welcome_uri, ) inserts.append(entry) for entry in inserts: # In the source tree, footer-generic.txt was renamed. entry['context'] = list_id connection.execute(template_table.insert().values(**entry)) with op.batch_alter_table('mailinglist') as batch_op: batch_op.drop_column('digest_footer_uri') batch_op.drop_column('digest_header_uri') batch_op.drop_column('footer_uri') batch_op.drop_column('header_uri') batch_op.drop_column('goodbye_message_uri') batch_op.drop_column('welcome_message_uri') with op.batch_alter_table('domain') as batch_op: batch_op.drop_column('base_url') def downgrade(): # Add back the original columns to the mailinglist table. for column in CONVERSION_MAPPING: if not exists_in_db(op.get_bind(), 'mailinglist', column): op.add_column( 'mailinglist', sa.Column(column, SAUnicode, nullable=True)) op.add_column('domain', sa.Column('base_url', SAUnicode)) # Put all the templates with a context mapping the list-id back into the # mailinglist table. No other contexts are supported, so just throw those # away. template_table = sa.sql.table( 'template', sa.sql.column('id', sa.Integer), sa.sql.column('name', SAUnicode), sa.sql.column('context', SAUnicode), sa.sql.column('uri', SAUnicode), sa.sql.column('username', SAUnicode), sa.sql.column('password', SAUnicode), ) mlist_table = sa.sql.table( 'mailinglist', sa.sql.column('id', sa.Integer), sa.sql.column('list_id', SAUnicode), sa.sql.column('digest_footer_uri', SAUnicode), sa.sql.column('digest_header_uri', SAUnicode), sa.sql.column('footer_uri', SAUnicode), sa.sql.column('header_uri', SAUnicode), sa.sql.column('goodbye_message_uri', SAUnicode), sa.sql.column('welcome_message_uri', SAUnicode), ) connection = op.get_bind() for (table_id, name, context, uri, username, password ) in connection.execute(template_table.select()).fetchall(): mlist = connection.execute(mlist_table.select().where( mlist_table.c.list_id == context)).fetchone() if mlist is None: continue attribute = REVERSE_MAPPING.get(name) if attribute is not None: connection.execute(mlist_table.update().where( mlist_table.c.list_id == context).values( **{attribute: uri})) op.drop_table('file_cache') op.drop_table('template') # Also delete the file cache directories. Don't delete the cache # directory itself though. for path in os.listdir(config.CACHE_DIR): full_path = os.path.join(config.CACHE_DIR, path) if os.path.isdir(full_path): shutil.rmtree(full_path) mailman-3.2.2/src/mailman/database/base.py0000644000175000017500000000747713442110351021520 0ustar maxkingmaxking00000000000000# Copyright (C) 2006-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Common database support.""" import logging from mailman.config import config from mailman.interfaces.database import IDatabase from mailman.utilities.string import expand from public import public from sqlalchemy import create_engine from sqlalchemy.orm import sessionmaker from zope.interface import implementer log = logging.getLogger('mailman.database') @public @implementer(IDatabase) class SABaseDatabase: """The database base class for use with SQLAlchemy. Use this as a base class for your DB-Specific derived classes. """ def __init__(self): self.url = None self.store = None def begin(self): """See `IDatabase`.""" # SQLAlchemy does this for us. pass def commit(self): """See `IDatabase`.""" self.store.commit() def abort(self): """See `IDatabase`.""" self.store.rollback() def _pre_reset(self, store): """Clean up method for testing. This method is called during the test suite just before all the model tables are removed. Override this to perform any database-specific pre-removal cleanup. """ pass def _post_reset(self, store): """Clean up method for testing. This method is called during the test suite just after all the model tables have been removed. Override this to perform any database-specific post-removal cleanup. """ pass def _prepare(self, url): """Prepare the database for creation. Some database backends need to do so me prep work before letting Storm create the database. For example, we have to touch the SQLite .db file first so that it has the proper file modes. """ pass def initialize(self, debug=None): """See `IDatabase`.""" # Calculate the engine url. url = expand(config.database.url, None, config.paths) self._prepare(url) log.debug('Database url: %s', url) # XXX By design of SQLite, database file creation does not honor # umask. See their ticket #1193: # http://www.sqlite.org/cvstrac/tktview?tn=1193,31 # # This sucks for us because the mailman.db file /must/ be group # writable, however even though we guarantee our umask is 002 here, it # still gets created without the necessary g+w permission, due to # SQLite's policy. This should only affect SQLite engines because its # the only one that creates a little file on the local file system. # This kludges around their bug by "touch"ing the database file before # SQLite has any chance to create it, thus honoring the umask and # ensuring the right permissions. We only try to do this for SQLite # engines, and yes, we could have chmod'd the file after the fact, but # half dozen and all... self.url = url self.engine = create_engine( url, isolation_level='READ UNCOMMITTED', pool_pre_ping=True) session = sessionmaker(bind=self.engine) self.store = session() self.store.commit() mailman-3.2.2/src/mailman/database/factory.py0000644000175000017500000001205713442110351022243 0ustar maxkingmaxking00000000000000# Copyright (C) 2012-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Database factory.""" import os import types import alembic.command from alembic.migration import MigrationContext from alembic.script import ScriptDirectory from flufl.lock import Lock from mailman.config import config from mailman.database.alembic import alembic_cfg from mailman.database.model import Model from mailman.interfaces.database import ( DatabaseError, IDatabase, IDatabaseFactory) from mailman.utilities.modules import call_name from public import public from sqlalchemy import MetaData from zope.interface import implementer from zope.interface.verify import verifyObject LAST_STORM_SCHEMA_VERSION = '20130406000000' @public @implementer(IDatabaseFactory) class DatabaseFactory: """Create a new database.""" @staticmethod def create(): """See `IDatabaseFactory`.""" with Lock(os.path.join(config.LOCK_DIR, 'dbcreate.lck')): database_class = config.database['class'] database = call_name(database_class) verifyObject(IDatabase, database) database.initialize() SchemaManager(database).setup_database() database.commit() return database @public class SchemaManager: "Manage schema migrations.""" def __init__(self, database): self._database = database self._script = ScriptDirectory.from_config(alembic_cfg) def _get_storm_schema_version(self): metadata = MetaData() metadata.reflect(bind=self._database.engine) if 'version' not in metadata.tables: # There are no Storm artifacts left. return None Version = metadata.tables['version'] last_version = self._database.store.query(Version.c.version).filter( Version.c.component == 'schema' ).order_by(Version.c.version.desc()).first() # Don't leave open transactions or they will block any schema change. self._database.commit() return last_version def setup_database(self): context = MigrationContext.configure(self._database.store.connection()) current_rev = context.get_current_revision() head_rev = self._script.get_current_head() if current_rev == head_rev: # We're already at the latest revision so there's nothing to do. return head_rev if current_rev is None: # No Alembic information is available. storm_version = self._get_storm_schema_version() if storm_version is None: # Initial database creation. Model.metadata.create_all(self._database.engine) self._database.commit() alembic.command.stamp(alembic_cfg, 'head') else: # The database was previously managed by Storm. if storm_version.version < LAST_STORM_SCHEMA_VERSION: raise DatabaseError( 'Upgrades skipping beta versions is not supported.') # Run migrations to remove the Storm-specific table and upgrade # to SQLAlchemy and Alembic. alembic.command.upgrade(alembic_cfg, 'head') elif current_rev != head_rev: alembic.command.upgrade(alembic_cfg, 'head') return head_rev def _reset(self): """See `IDatabase`.""" # Avoid a circular import at module level. from mailman.database.model import Model self.store.rollback() self._pre_reset(self.store) Model._reset(self) self._post_reset(self.store) self.store.commit() @public @implementer(IDatabaseFactory) class DatabaseTestingFactory: """Create a new database for testing.""" @staticmethod def create(): """See `IDatabaseFactory`.""" database_class = config.database['class'] database = call_name(database_class) verifyObject(IDatabase, database) database.initialize() # Remove existing tables (PostgreSQL will keep them across runs) metadata = MetaData(bind=database.engine) metadata.reflect() metadata.drop_all() database.commit() # Now create the current model without Alembic upgrades. Model.metadata.create_all(database.engine) database.commit() # Make _reset() a bound method of the database instance. database._reset = types.MethodType(_reset, database) return database mailman-3.2.2/src/mailman/database/helpers.py0000644000175000017500000000241313442110351022231 0ustar maxkingmaxking00000000000000# Copyright (C) 2015-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Common database helpers.""" import sqlalchemy as sa from public import public @public def is_sqlite(bind): return bind.dialect.name == 'sqlite' @public def is_mysql(bind): return bind.dialect.name == 'mysql' @public def exists_in_db(bind, tablename, columnname=None): md = sa.MetaData() md.reflect(bind=bind) if columnname is None: return tablename in md.tables else: return ( tablename in md.tables and columnname in [c.name for c in md.tables[tablename].columns] ) mailman-3.2.2/src/mailman/database/model.py0000644000175000017500000000371013442110351021670 0ustar maxkingmaxking00000000000000# Copyright (C) 2006-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Base class for all database classes.""" from contextlib import closing from mailman.config import config from public import public from sqlalchemy.ext.declarative import declarative_base class ModelMeta: """The custom metaclass for all model base classes. This is used in the test suite to quickly reset the database after each test. It works by iterating over all the tables, deleting each. The test suite will then recreate the tables before each test. """ @staticmethod def _reset(db): with closing(config.db.engine.connect()) as connection: transaction = connection.begin() try: # Delete all the tables in reverse foreign key dependency # order. # https://docs.sqlalchemy.org/en/latest/core/metadata.html \ # #accessing-tables-and-columns for table in reversed(Model.metadata.sorted_tables): connection.execute(table.delete()) except: # noqa: E722 pragma: nocover transaction.rollback() raise else: transaction.commit() Model = declarative_base(cls=ModelMeta) public(Model=Model) mailman-3.2.2/src/mailman/database/mysql.py0000644000175000017500000000235713442110351021743 0ustar maxkingmaxking00000000000000# Copyright (C) 2016-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """MySQL database support""" from mailman.database.base import SABaseDatabase from mailman.database.model import Model from public import public @public class MySQLDatabase(SABaseDatabase): """Database class for MySQL.""" def _post_reset(self, store): """Reset AUTO_INCREMENT counters for all the tables.""" super()._post_reset(store) tables = reversed(Model.metadata.sorted_tables) for table in tables: store.execute('ALTER TABLE {} AUTO_INCREMENT = 1;'.format(table)) mailman-3.2.2/src/mailman/database/postgresql.py0000644000175000017500000000372613442110351023002 0ustar maxkingmaxking00000000000000# Copyright (C) 2011-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """PostgreSQL database support.""" from mailman.database.base import SABaseDatabase from mailman.database.model import Model from public import public from sqlalchemy import Integer @public class PostgreSQLDatabase(SABaseDatabase): """Database class for PostgreSQL.""" def _post_reset(self, store): """PostgreSQL-specific test suite cleanup. Reset the _id_seq.last_value so that primary key ids restart from zero for new tests. """ super()._post_reset(store) tables = reversed(Model.metadata.sorted_tables) # Recipe adapted from # https://stackoverflow.com/questions/544791/ # django-postgresql-how-to-reset-primary-key for table in tables: for column in table.primary_key: if (column.autoincrement and isinstance(column.type, Integer) # noqa: W503 and not column.foreign_keys): # noqa: W503 store.execute("""\ SELECT setval('"{0}_{1}_seq"', coalesce(max("{1}"), 1), max("{1}") IS NOT null) FROM "{0}"; """.format(table, column.name)) mailman-3.2.2/src/mailman/database/sqlite.py0000644000175000017500000000274513442110351022100 0ustar maxkingmaxking00000000000000# Copyright (C) 2011-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """SQLite database support.""" import os from mailman.database.base import SABaseDatabase from public import public from urllib.parse import urlparse @public class SQLiteDatabase(SABaseDatabase): """Database class for SQLite.""" def _prepare(self, url): parts = urlparse(url) assert parts.scheme == 'sqlite', ( 'Database url mismatch (expected sqlite prefix): {0}'.format(url)) # Ensure that the SQLite database file has the proper permissions, # since SQLite doesn't play nice with umask. path = os.path.normpath(parts.path) fd = os.open( path, os.O_WRONLY | os.O_NONBLOCK | os.O_CREAT, 0o666) # Ignore errors if fd > 0: os.close(fd) mailman-3.2.2/src/mailman/database/tests/0000755000175000017500000000000013445614541021374 5ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/database/tests/__init__.py0000644000175000017500000000000013244427337023475 0ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/database/tests/test_factory.py0000644000175000017500000001545213442110351024446 0ustar maxkingmaxking00000000000000# Copyright (C) 2013-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Test database schema migrations""" import unittest import alembic.command from contextlib import suppress from mailman.config import config from mailman.database.alembic import alembic_cfg from mailman.database.factory import LAST_STORM_SCHEMA_VERSION, SchemaManager from mailman.database.helpers import is_mysql from mailman.database.model import Model from mailman.database.types import SAUnicode from mailman.interfaces.database import DatabaseError from mailman.testing.layers import ConfigLayer from sqlalchemy import Column, Integer, MetaData, Table from sqlalchemy.exc import OperationalError, ProgrammingError from sqlalchemy.schema import Index from unittest.mock import patch class TestSchemaManager(unittest.TestCase): layer = ConfigLayer def setUp(self): # Drop the existing model tables. Model.metadata.drop_all(config.db.engine) # Drop leftover tables (e.g. Alembic & Storm schema versions). md = MetaData() md.reflect(bind=config.db.engine) for table in md.sorted_tables: table.drop(config.db.engine) self.schema_mgr = SchemaManager(config.db) def tearDown(self): self._drop_storm_database() # Restore a virgin database. Model.metadata.create_all(config.db.engine) def _table_exists(self, tablename): md = MetaData() md.reflect(bind=config.db.engine) return tablename in md.tables def _create_storm_database(self, revision): version_table = Table( 'version', Model.metadata, Column('id', Integer, primary_key=True), Column('component', SAUnicode), Column('version', SAUnicode), ) version_table.create(config.db.engine) config.db.store.execute(version_table.insert().values( component='schema', version=revision)) config.db.commit() # Other Storm specific changes, those SQL statements hopefully work on # all DB engines... config.db.engine.execute( 'ALTER TABLE mailinglist ADD COLUMN acceptable_aliases_id INT') # In case of MySQL, you cannot create/drop indexes on primary keys # manually as it is handled automatically by MySQL. if not is_mysql(config.db.engine): Index('ix_user__user_id').drop(bind=config.db.engine) # Don't pollute our main metadata object, create a new one. md = MetaData() user_table = Model.metadata.tables['user'].tometadata(md) Index('ix_user_user_id', user_table.c._user_id).create( bind=config.db.engine) config.db.commit() def _drop_storm_database(self): """Remove the leftovers from a Storm DB. A drop_all() must be issued afterwards. """ if 'version' in Model.metadata.tables: version = Model.metadata.tables['version'] version.drop(config.db.engine, checkfirst=True) Model.metadata.remove(version) # If it's nonexistent, PostgreSQL raises a ProgrammingError while # SQLite raises an OperationalError. Since MySQL automatically handles # indexes for primary keys, don't try doing it with that backend. if not is_mysql(config.db.engine): with suppress(ProgrammingError, OperationalError): Index('ix_user_user_id').drop(bind=config.db.engine) config.db.commit() def test_current_database(self): # The database is already at the latest version. alembic.command.stamp(alembic_cfg, 'head') with patch('alembic.command') as alembic_command: self.schema_mgr.setup_database() self.assertFalse(alembic_command.stamp.called) self.assertFalse(alembic_command.upgrade.called) @patch('alembic.command.upgrade') def test_initial(self, alembic_command_upgrade): # No existing database. self.assertFalse(self._table_exists('mailinglist')) self.assertFalse(self._table_exists('alembic_version')) # For the initial setup of the database, the upgrade command will not # be called. The tables will be created and then the schema stamped # at Alembic's latest revision. head_rev = self.schema_mgr.setup_database() self.assertFalse(alembic_command_upgrade.called) self.assertTrue(self._table_exists('mailinglist')) self.assertTrue(self._table_exists('alembic_version')) # The current Alembic revision is the same as the initial revision. md = MetaData() md.reflect(bind=config.db.engine) current_rev = config.db.engine.execute( md.tables['alembic_version'].select()).scalar() self.assertEqual(current_rev, head_rev) @patch('alembic.command') def test_storm(self, alembic_command): # Existing Storm database. Model.metadata.create_all(config.db.engine) self._create_storm_database(LAST_STORM_SCHEMA_VERSION) self.schema_mgr.setup_database() self.assertFalse(alembic_command.stamp.called) self.assertTrue(alembic_command.upgrade.called) @patch('alembic.command') def test_old_storm(self, alembic_command): # Existing Storm database in an old version. Model.metadata.create_all(config.db.engine) self._create_storm_database('001') self.assertRaises(DatabaseError, self.schema_mgr.setup_database) self.assertFalse(alembic_command.stamp.called) self.assertFalse(alembic_command.upgrade.called) def test_old_db(self): # The database is in an old revision, must upgrade. alembic.command.stamp(alembic_cfg, 'head') md = MetaData() md.reflect(bind=config.db.engine) config.db.store.execute(md.tables['alembic_version'].delete()) config.db.store.execute(md.tables['alembic_version'].insert().values( version_num='dummyrevision')) config.db.commit() with patch('alembic.command') as alembic_command: self.schema_mgr.setup_database() self.assertFalse(alembic_command.stamp.called) self.assertTrue(alembic_command.upgrade.called) mailman-3.2.2/src/mailman/database/tests/test_migrations.py0000644000175000017500000005545613442110351025163 0ustar maxkingmaxking00000000000000# Copyright (C) 2015-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Test database schema migrations with Alembic""" import os import unittest import sqlalchemy as sa import alembic.command from mailman.app.lifecycle import create_list from mailman.config import config from mailman.database.alembic import alembic_cfg from mailman.database.helpers import exists_in_db from mailman.database.model import Model from mailman.database.transaction import transaction from mailman.database.types import Enum, SAUnicode from mailman.interfaces.action import Action from mailman.interfaces.cache import ICacheManager from mailman.interfaces.member import MemberRole from mailman.interfaces.template import ITemplateManager from mailman.interfaces.usermanager import IUserManager from mailman.testing.layers import ConfigLayer from warnings import catch_warnings, simplefilter from zope.component import getUtility class TestMigrations(unittest.TestCase): layer = ConfigLayer def setUp(self): alembic.command.stamp(alembic_cfg, 'head') def tearDown(self): # Drop and restore a virgin database. config.db.store.rollback() md = sa.MetaData(bind=config.db.engine) md.reflect() # We have circular dependencies between user and address, thus we can't # use drop_all() without getting a warning. Setting use_alter to True # on the foreign keys helps SQLAlchemy mark those loops as known. for tablename in ('user', 'address'): if tablename not in md.tables: continue for fk in md.tables[tablename].foreign_keys: fk.constraint.use_alter = True md.drop_all() Model.metadata.create_all(config.db.engine) def test_all_migrations(self): script_dir = alembic.script.ScriptDirectory.from_config(alembic_cfg) revisions = [sc.revision for sc in script_dir.walk_revisions()] with catch_warnings(): simplefilter('ignore', UserWarning) # Alembic/SQLite does not like something about these migrations. # They're more or less inconsequential in practice (since users # will rarely if ever downgrade their database), but it does # clutter up the test output, so just suppress the warning. # # E.g. # alembic/util/messaging.py:69: UserWarning: # Skipping unsupported ALTER for creation of implicit constraint for revision in revisions: alembic.command.downgrade(alembic_cfg, revision) revisions.reverse() for revision in revisions: alembic.command.upgrade(alembic_cfg, revision) def test_42756496720_header_matches(self): test_header_matches = [ ('test-header-1', 'test-pattern-1'), ('test-header-2', 'test-pattern-2'), ('test-header-3', 'test-pattern-3'), ] mlist_table = sa.sql.table( 'mailinglist', sa.sql.column('id', sa.Integer), sa.sql.column('header_matches', sa.PickleType) ) header_match_table = sa.sql.table( 'headermatch', sa.sql.column('mailing_list_id', sa.Integer), sa.sql.column('header', SAUnicode), sa.sql.column('pattern', SAUnicode), ) # Bring the DB to the revision that is being tested. alembic.command.downgrade(alembic_cfg, '42756496720') # Test downgrading. config.db.store.execute(mlist_table.insert().values(id=1)) config.db.store.execute(header_match_table.insert().values( [{'mailing_list_id': 1, 'header': hm[0], 'pattern': hm[1]} for hm in test_header_matches])) config.db.store.commit() alembic.command.downgrade(alembic_cfg, '2bb9b382198') results = config.db.store.execute( mlist_table.select()).fetchall() self.assertEqual(results[0].header_matches, test_header_matches) self.assertFalse(exists_in_db(config.db.engine, 'headermatch')) config.db.store.commit() # Test upgrading. alembic.command.upgrade(alembic_cfg, '42756496720') results = config.db.store.execute( header_match_table.select()).fetchall() self.assertEqual( results, [(1, hm[0], hm[1]) for hm in test_header_matches]) def test_47294d3a604_pendable_keyvalues(self): # We have 5 pended items: # - one is a probe request # - one is a subscription request # - one is a moderation request # - one is a held message # - one is a registration request in the new format # # The first three used to have no 'type' key and must be properly # typed, the held message used to have a type key, but in JSON, and # must be converted. pended_table = sa.sql.table( 'pended', sa.sql.column('id', sa.Integer), ) keyvalue_table = sa.sql.table( 'pendedkeyvalue', sa.sql.column('id', sa.Integer), sa.sql.column('key', SAUnicode), sa.sql.column('value', SAUnicode), sa.sql.column('pended_id', sa.Integer), ) def get_from_db(): # noqa: E306 results = {} for i in range(1, 6): query = sa.sql.select( [keyvalue_table.c.key, keyvalue_table.c.value] ).where( keyvalue_table.c.pended_id == i ) results[i] = dict([ (r['key'], r['value']) for r in config.db.store.execute(query).fetchall() ]) return results # Start at the previous revision with transaction(): alembic.command.downgrade(alembic_cfg, '33bc0099223') for i in range(1, 6): config.db.store.execute(pended_table.insert().values(id=i)) config.db.store.execute(keyvalue_table.insert().values([ {'pended_id': 1, 'key': 'member_id', 'value': 'test-value'}, {'pended_id': 2, 'key': 'token_owner', 'value': 'test-value'}, {'pended_id': 3, 'key': '_mod_message_id', 'value': 'test-value'}, {'pended_id': 4, 'key': 'type', 'value': '"held message"'}, {'pended_id': 5, 'key': 'type', 'value': 'registration'}, ])) # Upgrading. with transaction(): alembic.command.upgrade(alembic_cfg, '47294d3a604') results = get_from_db() for i in range(1, 5): self.assertIn('type', results[i]) self.assertEqual(results[1]['type'], 'probe') self.assertEqual(results[2]['type'], 'subscription') self.assertEqual(results[3]['type'], 'data') self.assertEqual(results[4]['type'], 'held message') self.assertEqual(results[5]['type'], 'registration') # Downgrading. with transaction(): alembic.command.downgrade(alembic_cfg, '33bc0099223') results = get_from_db() for i in range(1, 4): self.assertNotIn('type', results[i]) self.assertEqual(results[4]['type'], '"held message"') self.assertEqual(results[5]['type'], '"registration"') def test_70af5a4e5790_digests(self): IDS_TO_DIGESTABLE = [ (1, True), (2, False), (3, False), (4, True), ] mlist_table = sa.sql.table( 'mailinglist', sa.sql.column('id', sa.Integer), sa.sql.column('digests_enabled', sa.Boolean) ) # Downgrading. with transaction(): for table_id, enabled in IDS_TO_DIGESTABLE: config.db.store.execute(mlist_table.insert().values( id=table_id, digests_enabled=enabled)) with transaction(): alembic.command.downgrade(alembic_cfg, '47294d3a604') results = config.db.store.execute( 'SELECT id, digestable FROM mailinglist').fetchall() self.assertEqual(results, IDS_TO_DIGESTABLE) # Upgrading. with transaction(): alembic.command.upgrade(alembic_cfg, '70af5a4e5790') results = config.db.store.execute( 'SELECT id, digests_enabled FROM mailinglist').fetchall() self.assertEqual(results, IDS_TO_DIGESTABLE) def test_70af5a4e5790_data_paths(self): # Create a couple of mailing lists through the standard API. with transaction(): create_list('ant@example.com') create_list('bee@example.com') # Downgrade and verify that the old data paths exist. alembic.command.downgrade(alembic_cfg, '47294d3a604') self.assertTrue(os.path.exists( os.path.join(config.LIST_DATA_DIR, 'ant@example.com'))) self.assertTrue(os.path.exists( os.path.join(config.LIST_DATA_DIR, 'ant@example.com'))) # Upgrade and verify that the new data paths exists and the old ones # no longer do. alembic.command.upgrade(alembic_cfg, '70af5a4e5790') self.assertFalse(os.path.exists( os.path.join(config.LIST_DATA_DIR, 'ant@example.com'))) self.assertFalse(os.path.exists( os.path.join(config.LIST_DATA_DIR, 'bee@example.com'))) self.assertTrue(os.path.exists( os.path.join(config.LIST_DATA_DIR, 'ant.example.com'))) self.assertTrue(os.path.exists( os.path.join(config.LIST_DATA_DIR, 'bee.example.com'))) def test_7b254d88f122_moderation_action(self): # Create a mailing list through the standard API. with transaction(): ant = create_list('ant@example.com') default_member_action = ant.default_member_action default_nonmember_action = ant.default_nonmember_action sa.sql.table( 'mailinglist', sa.sql.column('id', sa.Integer), sa.sql.column('list_id', SAUnicode), sa.sql.column('default_member_action', Enum(Action)), sa.sql.column('default_nonmember_action', Enum(Action)), ) member_table = sa.sql.table( 'member', sa.sql.column('id', sa.Integer), sa.sql.column('list_id', SAUnicode), sa.sql.column('address_id', sa.Integer), sa.sql.column('role', Enum(MemberRole)), sa.sql.column('moderation_action', Enum(Action)), ) user_manager = getUtility(IUserManager) with transaction(): # Start at the previous revision. alembic.command.downgrade(alembic_cfg, 'd4fbb4fd34ca') # Create some members. anne = user_manager.create_address('anne@example.com') bart = user_manager.create_address('bart@example.com') cris = user_manager.create_address('cris@example.com') dana = user_manager.create_address('dana@example.com') # Flush the database to get the last auto-increment id. config.db.store.flush() # Assign some moderation actions to the members created above. config.db.store.execute(member_table.insert().values([ {'address_id': anne.id, 'role': MemberRole.owner, 'list_id': 'ant.example.com', 'moderation_action': Action.accept}, {'address_id': bart.id, 'role': MemberRole.moderator, 'list_id': 'ant.example.com', 'moderation_action': Action.accept}, {'address_id': cris.id, 'role': MemberRole.member, 'list_id': 'ant.example.com', 'moderation_action': Action.defer}, {'address_id': dana.id, 'role': MemberRole.nonmember, 'list_id': 'ant.example.com', 'moderation_action': Action.hold}, ])) # Cris and Dana have actions which match the list default action for # members and nonmembers respectively. query = sa.sql.select([member_table.c.moderation_action]).where( sa.and_( member_table.c.role == MemberRole.member, member_table.c.list_id == 'ant.example.com', # cris@example.com member_table.c.address_id == 3, ) ) action = config.db.store.execute(query).fetchone()[0] self.assertEqual(action, default_member_action) query = sa.sql.select([member_table.c.moderation_action]).where( sa.and_( member_table.c.role == MemberRole.nonmember, member_table.c.list_id == 'ant.example.com', # dana@example.com member_table.c.address_id == 4, ) ) action = config.db.store.execute(query).fetchone()[0] self.assertEqual(action, default_nonmember_action) # Upgrade and check the moderation_actions. Cris's and Dana's # actions have been set to None to fall back to the list defaults. alembic.command.upgrade(alembic_cfg, '7b254d88f122') members = config.db.store.execute(sa.select([ member_table.c.address_id, member_table.c.moderation_action, ])).fetchall() self.assertEqual(members, [ (anne.id, Action.accept), (bart.id, Action.accept), (cris.id, None), (dana.id, None), ]) # Downgrade and check that Cris's and Dana's actions have been set # explicitly. alembic.command.downgrade(alembic_cfg, 'd4fbb4fd34ca') members = config.db.store.execute(sa.select([ member_table.c.address_id, member_table.c.moderation_action, ])).fetchall() self.assertEqual(members, [ (anne.id, Action.accept), (bart.id, Action.accept), (cris.id, Action.defer), (dana.id, Action.hold), ]) def test_fa0d96e28631_upgrade_uris(self): mlist_table = sa.sql.table( 'mailinglist', sa.sql.column('id', sa.Integer), sa.sql.column('list_id', sa.Unicode), sa.sql.column('digest_footer_uri', sa.Unicode), sa.sql.column('digest_header_uri', sa.Unicode), sa.sql.column('footer_uri', sa.Unicode), sa.sql.column('header_uri', sa.Unicode), sa.sql.column('goodbye_message_uri', sa.Unicode), sa.sql.column('welcome_message_uri', sa.Unicode), ) with transaction(): # Start at the previous revision. alembic.command.downgrade(alembic_cfg, '7b254d88f122') # Create a mailing list with some URIs. config.db.store.execute(mlist_table.insert().values( list_id='ant.example.com', digest_footer_uri='mailman:///digest_footer.txt', digest_header_uri='mailman:///digest_header.txt', footer_uri='mailman:///footer.txt', header_uri='mailman:///header.txt', goodbye_message_uri='mailman:///goodbye.txt', welcome_message_uri='mailman:///welcome.txt', )) # Now upgrade and check to see if the values got into the template # table correctly. alembic.command.upgrade(alembic_cfg, 'fa0d96e28631') seen_names = [] template_table = sa.sql.table( 'template', sa.sql.column('id', sa.Integer), sa.sql.column('name', sa.Unicode), sa.sql.column('context', sa.Unicode), sa.sql.column('uri', sa.Unicode), sa.sql.column('username', sa.Unicode), sa.sql.column('password', sa.DateTime), ) for (table_id, name, context, uri, username, password) in config.db.store.execute( template_table.select()): # This information isn't available in the old database # version, so there's no way these can be set. seen_names.append(name) self.assertIsNone(username) self.assertIsNone(password) self.assertEqual(context, 'ant.example.com') self.assertEqual(uri, 'mailman:///{}.txt'.format({ 'list:digest:footer': 'digest_footer', 'list:digest:header': 'digest_header', 'list:regular:footer': 'footer', 'list:regular:header': 'header', 'list:user:notice:goodbye': 'goodbye', 'list:user:notice:welcome': 'welcome', }.get(name, name))) self.assertEqual(sorted(seen_names), [ 'list:digest:footer', 'list:digest:header', 'list:regular:footer', 'list:regular:header', 'list:user:notice:goodbye', 'list:user:notice:welcome', ]) def test_fa0d96e28631_upgrade_no_uris(self): mlist_table = sa.sql.table( 'mailinglist', sa.sql.column('id', sa.Integer), sa.sql.column('list_id', sa.Unicode), sa.sql.column('digest_footer_uri', sa.Unicode), sa.sql.column('digest_header_uri', sa.Unicode), sa.sql.column('footer_uri', sa.Unicode), sa.sql.column('header_uri', sa.Unicode), sa.sql.column('goodbye_message_uri', sa.Unicode), sa.sql.column('welcome_message_uri', sa.Unicode), ) # None of the URL parameters are defined. with transaction(): # Start at the previous revision. alembic.command.downgrade(alembic_cfg, '7b254d88f122') # Create a mailing list without any URLs. config.db.store.execute(mlist_table.insert().values( list_id='ant.example.com')) # Now upgrade. There are no templates. alembic.command.upgrade(alembic_cfg, 'fa0d96e28631') template_table = sa.sql.table( 'template', sa.sql.column('id', sa.Integer), ) entries = list(config.db.store.execute(template_table.select())) self.assertEqual(len(entries), 0) def test_fa0d96e28631_downgrade_uris(self): # Create some cache directory entries. self.assertTrue(os.path.exists(config.CACHE_DIR)) getUtility(ICacheManager).add('abc', 'def') self.assertNotEqual(len(os.listdir(config.CACHE_DIR)), 0) # Set up the templates using the current API. with transaction(): create_list('ant@example.com') manager = getUtility(ITemplateManager) manager.set('list:digest:footer', 'ant.example.com', 'mailman:///digest_footer.txt') manager.set('list:digest:header', 'ant.example.com', 'mailman:///digest_header.txt') manager.set('list:regular:footer', 'ant.example.com', 'mailman:///footer.txt') manager.set('list:regular:header', 'ant.example.com', 'mailman:///header.txt') manager.set('list:user:notice:welcome', 'ant.example.com', 'mailman:///welcome.txt') manager.set('list:user:notice:goodbye', 'ant.example.com', 'mailman:///goodbye.txt') mlist_table = sa.sql.table( 'mailinglist', sa.sql.column('id', sa.Integer), sa.sql.column('list_id', sa.Unicode), sa.sql.column('digest_footer_uri', sa.Unicode), sa.sql.column('digest_header_uri', sa.Unicode), sa.sql.column('footer_uri', sa.Unicode), sa.sql.column('header_uri', sa.Unicode), sa.sql.column('goodbye_message_uri', sa.Unicode), sa.sql.column('welcome_message_uri', sa.Unicode), ) alembic.command.downgrade(alembic_cfg, '7b254d88f122') for (table_id, list_id, digest_footer_uri, digest_header_uri, footer_uri, header_uri, goodbye_message_uri, welcome_message_uri) in config.db.store.execute( mlist_table.select()): self.assertEqual(list_id, 'ant.example.com') self.assertEqual(digest_footer_uri, 'mailman:///digest_footer.txt') self.assertEqual(digest_header_uri, 'mailman:///digest_header.txt') self.assertEqual(footer_uri, 'mailman:///footer.txt') self.assertEqual(header_uri, 'mailman:///header.txt') self.assertEqual(welcome_message_uri, 'mailman:///welcome.txt') self.assertEqual(goodbye_message_uri, 'mailman:///goodbye.txt') # The cache directories are gone too. self.assertEqual(len(os.listdir(config.CACHE_DIR)), 0, os.listdir(config.CACHE_DIR)) def test_fa0d96e28631_downgrade_missing_list(self): with transaction(): manager = getUtility(ITemplateManager) manager.set('list:regular:footer', 'missing.example.com', 'mailman:///missing-footer.txt') alembic.command.downgrade(alembic_cfg, '7b254d88f122') mlist_table = sa.sql.table( 'mailinglist', sa.sql.column('id', sa.Integer), sa.sql.column('footer_uri', sa.Unicode), ) self.assertEqual( len(list(config.db.store.execute(mlist_table.select()))), 0) def test_15401063d4e3_upgrade_if_exists(self): mlist_table = sa.sql.table( 'mailinglist', sa.sql.column('member_roster_visibility', sa.Integer), sa.sql.column('list_id', sa.Unicode)) with transaction(): # Start at the previous revision. alembic.command.downgrade(alembic_cfg, '15401063d4e3') # Create a mailing list with some existing column. config.db.store.execute(mlist_table.insert().values( list_id='ant.example.com', member_roster_visibility=2)) # Test that if the database already has member_roster_visibility filed, # then make sure that we can ugprade. alembic.command.upgrade(alembic_cfg, '15401063d4e3') mailman-3.2.2/src/mailman/database/transaction.py0000644000175000017500000000532213442110351023116 0ustar maxkingmaxking00000000000000# Copyright (C) 2006-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Transactional support.""" from contextlib import contextmanager from mailman.config import config from public import public @public @contextmanager def transaction(): """Context manager for ensuring the transaction is complete.""" try: yield except: # noqa: E722 config.db.abort() raise else: config.db.commit() @public def transactional(function): """Decorator for transactional support. When the function this decorator wraps exits cleanly, the current transaction is committed. When it exits uncleanly (i.e. because of an exception, the transaction is aborted. Either way, the current transaction is completed. """ def wrapper(*args, **kws): try: rtn = function(*args, **kws) config.db.commit() return rtn except: # noqa: E722 pragma: nocover config.db.abort() raise return wrapper @public @contextmanager def flush(): """Context manager for flushing SQLAlchemy. We need this for SA whereas we didn't need it for Storm because the latter did auto-reloads. However, in SA this is needed when we add or delete objects from the database. Use it when you need the id after adding, or when you want to be sure the object won't be found after a delete. This is lighter weight than committing the transaction. """ yield config.db.store.flush() @public def dbconnection(function): """Decorator for getting at the database connection. Use this to avoid having to access the global `config.db.store` attribute. This calls the function with `store` as the first argument. """ def wrapper(*args, **kws): # args[0] is self, if there is one. if len(args) > 0: return function(args[0], config.db.store, *args[1:], **kws) else: return function(config.db.store, **kws) return wrapper mailman-3.2.2/src/mailman/database/types.py0000644000175000017500000001076013442110351021737 0ustar maxkingmaxking00000000000000# Copyright (C) 2007-2019 by the Free Software Foundation, Inc. # # This file is part of GNU Mailman. # # GNU Mailman is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free # Software Foundation, either version 3 of the License, or (at your option) # any later version. # # GNU Mailman is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # GNU Mailman. If not, see . """Database type conversions.""" import uuid from public import public from sqlalchemy import Integer from sqlalchemy.dialects import postgresql from sqlalchemy.ext.compiler import compiles from sqlalchemy.types import CHAR, TypeDecorator, Unicode @public class Enum(TypeDecorator): """Handle Python 3.4 style enums. Stores an integer-based Enum as an integer in the database, and converts it on-the-fly. """ impl = Integer def __init__(self, enum, *args, **kw): super().__init__(*args, **kw) self.enum = enum def process_bind_param(self, value, dialect): if value is None: return None return value.value def process_result_value(self, value, dialect): if value is None: return None return self.enum(value) @public class UUID(TypeDecorator): """Platform-independent GUID type. Uses Postgresql's UUID type, otherwise uses CHAR(32), storing as stringified hex values. """ impl = CHAR def load_dialect_impl(self, dialect): if dialect.name == 'postgresql': return dialect.type_descriptor(postgresql.UUID()) else: return dialect.type_descriptor(CHAR(32)) def process_bind_param(self, value, dialect): if value is None: return value elif dialect.name == 'postgresql': return str(value) else: if not isinstance(value, uuid.UUID): value = uuid.UUID(value) return '%.32x' % value.int def process_result_value(self, value, dialect): if value is None: return value else: return uuid.UUID(value) @public class SAUnicode(TypeDecorator): """Unicode datatype to support fixed length VARCHAR in MySQL. This type compiles to VARCHAR(255) in case of MySQL, and in case of other dailects defaults to the Unicode type. This was created so that we don't have to alter the output of the default Unicode data type and it can still be used if needed in the codebase. """ impl = Unicode @compiles(SAUnicode) def default_sa_unicode(element, compiler, **kw): return compiler.visit_unicode(element, **kw) @compiles(SAUnicode, 'mysql') def compile_sa_unicode(element, compiler, **kw): # We hardcode the collate here to make string comparison case sensitive. return 'VARCHAR(255) COLLATE utf8_bin' @public class SAUnicodeLarge(TypeDecorator): """Similar to SAUnicode type, but compiles to VARCHAR(510). This is double size of SAUnicode defined above. """ impl = Unicode @compiles(SAUnicodeLarge, 'mysql') def compile_sa_unicode_large(element, compiler, **kw): # We hardcode the collate here to make string comparison case sensitive. return 'VARCHAR(510) COLLATE utf8_bin' @compiles(SAUnicodeLarge) def default_sa_unicode_large(element, compiler, **kw): return compiler.visit_unicode(element, **kw) @public class SAUnicodeXL(TypeDecorator): """Similar to SAUnicode type, but compiles to VARCHAR(20000). This allows very long values up to 20,000 bytes for cases where SAUnicodeLarge is not enough. Because of MySQL issues columns of this type should either not be indexed or should be indexed with an Index() function specifying mysql_length=N where N is (probably much) less than 767. See https://docs.sqlalchemy.org/en/latest/dialects/mysql.html#index-length """ impl = Unicode @compiles(SAUnicodeXL, 'mysql') def compile_sa_unicode_xl(element, compiler, **kw): # We hardcode the collate here to make string comparison case sensitive. return 'VARCHAR(20000) COLLATE utf8_bin' # pragma: nocover @compiles(SAUnicodeXL) def default_sa_unicode_xl(element, compiler, **kw): return compiler.visit_unicode(element, **kw) mailman-3.2.2/src/mailman/docs/0000755000175000017500000000000013445614541017416 5ustar maxkingmaxking00000000000000mailman-3.2.2/src/mailman/docs/8-miles-high.rst0000644000175000017500000001665313442110351022341 0ustar maxkingmaxking00000000000000======================================== Notes from the PyCon 2012 Mailman Sprint ======================================== .. authorship The notes are based on Barry Warsaw's description of the Mailman 3 pipeline at the PyCon 2012 Mailman sprint on March 13, with diagrams from his "Mailman" presentation at PyCon 2012. Transcribed by Stephen Turnbull. *These are notes from the Mailman sprint at PyCon 2012. They are not terribly well organized, nor fully fleshed out. Please edit and push branches to `Gitlab`_ or post patches to the Mailman bug tracker at .* The intent of this document is to provide a view of Mailman 3's workflow and structures from "eight miles high". Basic Messaging Handling Workflow ================================= Mailman accepts a message via the LMTP protocol (RFC 2033). It implements a simple LMTP server internally based on the LMTP server provided in the Python standard library. The LMTP server's responsibility is to parse the message into a tuple (*mlist*, *msg*, *msgdata*). If the parse fails (including messages which Mailman considers to be invalid due to lack of `Message-Id` as strongly recommended by RFC 2822 and RFC 5322), the message will be rejected, otherwise the parsed message and metadata dictionary are pickled, and the resulting *message pickle* added to one of the `in`, `command`, or `bounce` processing queues. .. graphviz:: digraph msgflow { rankdir = LR; node [shape=box, color=lightblue, style=filled]; msg [shape=ellipse, color=black, fillcolor=white]; lmtpd [label="LMTP\nSERVER"]; rts [label="Return\nto Sender"]; msg -> MTA [label="SMTP"]; MTA -> lmtpd [label="LMTP"]; lmtpd -> MTA [label="reject"]; lmtpd -> IN -> PIPELINE [label=".pck"]; IN -> rts; lmtpd -> BOUNCES [label=".pck"]; lmtpd -> COMMAND [label=".pck"]; } The `in` queue is processed by *filter chains* (explained below) to determine whether the post (or administrative request) will be processed. If not allowed, the message pickle is discarded, rejected (returned to sender), or held (saved for moderator approval -- not shown). Otherwise the message is added to the `pipeline` (i.e. posting) queue. (Note that rejecting at this stage is *not* equivalent to rejecting during LMTP processing. This issue is currently unresolved.) Each of the `command`, `bounce`, and `pipeline` queues is processed by a *pipeline of handlers* as in Mailman 2's pipeline. (Some functions such as spam detection that were handled in the Mailman 2 pipeline are now in the filter chains.) Handlers may copy messages to other queues (*e.g.*, `archive`), and eventually posted messages for distribution to the list membership end up in the `out` queue for injection into the MTA. The `virgin` queue (not depicted above) is a special queue for messages created by Mailman. .. graphviz:: digraph pipeline { node [shape=box, style=rounded, group=0] { "MIME\ndelete" -> "cleanse headers" -> "add headers" -> "calculate\nrecipients" -> "to digest" -> "to archive" -> "to outgoing" } node [shape=box, color=lightblue, style=filled, group=1] { rank=same; PIPELINE -> "MIME\ndelete" } { rank=same; "to digest" -> DIGEST } { rank=same; "to archive" -> ARCHIVE } { rank=same; "to outgoing" -> OUT } } Message Filtering ================= Once a message has been classified as a post or administrivia, rules are applied to determine whether the message should be distributed or acted on. Rules include things like "if the message's sender is a non-member, hold it for moderation", or "if the message contains an `Approved` header with a valid password, allow it to be posted". A rule may also make no decision, in which case message processing is passed on to the next rule in the filter chain. The default set of rules looks something like this: .. graphviz:: digraph chain_rules { rankdir=LR; /* This gives the right orientation of the columns. */ rank=same; subgraph in { IN [shape=box, color=lightblue, style=filled]; } subgraph rules { rankdir=TB; node [shape=record]; approved [label=" approved | { no | }"]; emergency [label=" emergency | { no | }"]; loop [label=" loop | { no | }"]; modmember [label=" member\nmoderated | { no | }"]; administrivia [group="0", label=" administrivia | { no | }"]; maxsize [label=" max\ size | { no | }"]; any [label=" any | { | }"]; truth [label=" truth | "]; } subgraph queues { rankdir=TB; node [shape=box, style=filled]; DISCARD [shape=invhouse, color=black, style=solid]; MODERATION [color=wheat]; HOLD [color=wheat]; action [color=wheat]; } { PIPELINE [shape=box, style=filled, color=cyan]; } IN -> approved:in; approved:no -> emergency:in [weight="100"]; approved:yes -> PIPELINE [minlen=2]; emergency:no -> loop:in; emergency:yes -> HOLD; loop:no -> modmember:in; loop:yes -> DISCARD; modmember:no -> administrivia:in; modmember:yes -> MODERATION; administrivia:no -> maxsize:in; administrivia:yes -> action; maxsize:no -> any:in; maxsize:yes -> MODERATION; any:no -> truth:in; any:yes -> MODERATION; truth:always -> PIPELINE [minlen=2]; } Configuration ============= Mailman 3 uses `lazr.config`_, essentially an "ini"-style configuration format. Each Runner's configuration object knows whether it should be started when the Mailman daemon starts, and what queue the Runner manages. Shell Commands ============== `mailman`: This is an ubercommand, with subcommands for all the various things admins might want to do, similar to Mailman 2's mailmanctl, but with more functionality. `bin/master`: The runner manager: starts, watches, stops the runner daemons. `bin/runner`: Individual runner daemons. Each instance is configured with arguments specified on the command line. User Model ========== A *user* represents a person. A user has an *id* and a *display name*, and optionally a list of linked addresses. Each *address* is a separate object, linked to no more than one user. A list *member* associates an address with a mailing list. Each list member has an id, a mailing list name, an address (which may be `None`, representing the user's *preferred address*), a list of preferences, and a *role* such as "owner" or "moderator". Roles are used to determine what kinds of mail the user receives via that membership. *Owners* will receive mail to *list*-owner, but not posts and moderation traffic, for example. A user with multiple roles on a single list will therefore have multiple memberships in that list, one for each role. Roles are implemented by "magical, invisible" *rosters* which are objects representing queries on the membership database. List Styles =========== Each list *style* is a named object. Its attributes are functions used to apply the relevant style settings to the mailing list *at creation time*. Since these are functions, they can be composed in various ways, to create substyles, *etc*. .. _`lazr.config`: https://pypi.org/project/lazr.config/ .. _`Gitlab` : https://gitlab.com/mailman/mailman mailman-3.2.2/src/mailman/docs/ACKNOWLEDGMENTS.rst0000644000175000017500000001501313421245331022364 0ustar maxkingmaxking00000000000000.. -*- coding: utf-8 -*- =========================== GNU Mailman Acknowledgments =========================== Copyright (C) 1998-2018 by the Free Software Foundation, Inc. Governance ========== GNU Mailman was invented by John Viega. Barry Warsaw is the current project leader. Aurélien Bompard leads HyperKitty and bundler development. Florian Fuchs leads Postorius development. Development of mailman.client is a group effort. All project decisions are made by consensus via the Mailman Cabal, er, Steering Committee which can be contacted directly via mailman-cabal@python.org Core Developers =============== The following folks are or have been core developers of Mailman (in alphabetical order): * Abhilash Raj, Mailman's Youngest Core Dev * Aurélien Bompard, Pizzaman * Barry Warsaw, Mailman's yappy guard dog * Florian Fuchs * Harald Meland, Norse Mailman * John Viega, Mailman's inventor * Ken Manheimer, Mailman's savior * Mark Sapiro, Mailman's compulsive responder * Scott Cotton, Cookie-Monster * Stephen J. Turnbull, Standards Otaku * Terri Oda * Thomas Wouters, Mailman's Dutch treat * Tokio Kikuchi (RIP), Mailman's weatherman Copyright Assignees =================== Here is the list of other contributors who have donated large bits of code, and have assigned copyright for contributions to the FSF: * Andreas Schosser * Andrija Arsic * Ben Gertzfield * Cedric Knight * Claudia Schmidt * Ethan Mindlace Fremen * Jimmy Bergman * Joe Dugan * Juan Carlos Rey Anaya * Les Niles * Mads Kiilerich * Norbert Bollow * Patrick Ben Koetter * Reed O'Brien * Richard Barrett * Richard Wackerbarth * Simon Hanna * Simone Piunno * Stephan Berndts * Stephen Goss * The Dragon De Monsyne * Victoriano Giralt Special Thanks ============== Very special thanks to Andrija Arsic for his winning GNU Mailman logos. Thanks also go to the following people for their important contributions in other aspects of the Mailman project: * Brad Knowles * Clytie Siddall * JC Dill Thanks also to Dragon for his original Mailman logo contribution, and to Terri Oda for the neat shortcut icon and the member documentation. Control.com sponsored development of several Mailman 2.1 features, including topics filters, external membership sources, and initial virtual mailing list support. My thanks especially to Dan Pierson and Ken Crater from Control.com. Here is the list of other people who have contributed useful ideas, suggestions, bug fixes, testing, etc., or who have been very helpful in answering questions on mailman-users. Please let me know if you have been left off the list! * "office" * Ademar de Souza Reis, Jr. * Alessio Bragadini * Alexander Sulfrian * Andrew Kuchling * Andrew Martynov * Anti Veeranna * Anton Antonov * Antonis Limperis * Ashley M. Kirchner * Balazs Nagy * Bartosz Sawicki * Ben Burnett * Bernhard Reiter * Bernhard Schmidt * Bert Hubert * Bill Wagner * Blair Zajac * Bob Fleck * Bob Puff * Bojan * Cabel Sasser * Carson Gaspar * Chris Kolar * Chris Pepper * Chris Ryan * Chris Snell * Christian F Buser * Christian Reis * Christopher P. Lindsey * Chuq Von Rospach * Claudio Cattazzo * Dai Xiaoguang * Dale Newfield * Dale Stimson * Dan Mick * Dan Ohnesorg * Dan Wilder * Daniel Buchmann * Daniel Zeiss * Danil Smirnov * Danny Terweij * Dario Lopez-Kästen * Darrell Fuhriman * David Abrahams * David B. O'Donnell * David Blomquist * David Champion * David Gibbs * David Habben * David Martínez Moreno * David Soto * David T-G * Diego Francisco de Gastal Morales * Dirk Mueller * Dmitri I GOULIAEV * Don Porter * Donn Cave * Ed Lau * Eddie Kohler * Egon Frerich * Emerson Ribeiro de Mello * Emilio Delgado * Eric D. Christensen * Erik Forsberg * Erik Myllymaki * Eva Österlind * Fabian Wenk * Federico Grau * Fil * Florian Weimer * Francesco Potortì * Francis Jorissen * Franck Martin * Fred Drake * Gabriel P. Silva * Garey Mills * Gari Araolaza * Geoff Mayes * Gerald Oskoboiny * Gergely Madarasz * Gleydson Mazioli da Silva * Grant Bowman * Greg Lindahl * Greg Stein * Greg Ward * Guido van Rossum * Harald Koch * Heiko Rommel * Henny Huisman * Hrvoje Niksic * Hugo Koji Kobayashi * Hye-Shik Chang * Ikeda Soji * J C Lawrence * \J. D. Bronson * James Henstridge * Jan Veuger * Jason R. Mastaler * Javad Hoseini * Javier Rial Rodríguez * Jay Luker * Jeff Berliner * Jeff Hahn * Jens Vagelpohl * Jeremy Hylton * Jim Popovitch * Jim Tittsler * Jimmy Bergman * Joe Peterson * John A. Martin * John Carnes * John Dennis * John Read * Jon Parise * Jonas Muerer * Jonas Smedegaard * Joni Töyrylä * Jose Paulo Moitinho de Almeida * Julio A. Cartaya * Kai Schaetzl * Karl Chen * Karoly Segesdi * Kathleen Webb * Kerem Erkan * Kleber A. Benatti * L'homme Moderne * Les Niles * Lindsay Haisley * Lionel Elie Mamane * Luca Maranzano * Luigi Rosa * Mahyar Moghimi * Marc MERLIN * Marcos Costales * Mark Weaver * Martijn Dekker * Martin 'Joey' Schulze * Martin Matuska * Martin Mokrejs * Martin Pool * Martin von Loewis * Matthias Andree * Matthias Juchem * Matthias Klose * Maxim Dzumanenko * Maxime Carron * Maximillian Dornseif * Mentor Cana * Michael Fischer v. Mollard * Michael Mclay * Michael Meltzer * Michael Ranner * Michael Yount * Mike Avery * Mike Noyes * Mikhail Sobolev * Mikhail Zabaluev * Miloslav Trmac * Mirian Margiani * Moreno Baricevic * Moritz Naumann * Ned Dawes * Nicholas Russo * Nigel Metheringham * Nino Katic * Noam Zeilberger * Ousmane Wilane * Owen Taylor * Pascal GEORGE * Pasi Sjöholm * Patrick Finnerty * Patrick Koetter * Paul Cox * Paul Hebble * Peer Heinlein * Pekka Haavisto * Phil Pennock * Piarres Beobide Egaña * PieterB * Ping Yeh * Ralf Doeblitz * Ralf Hildebrandt * Ricardo Kustner * Rob Ellis * Robert Daeley * Robert Garrigós * Rodolfo Pilas * Roger Tsang * Ron Jarrell * Rostyk Ivantsiv * SATOH Fumiyasu * SHIGENO Kazutaka * Sean Reifschneider * Seb Wills * Skye Poier * Stan Bubrouski * Stefan Divjak * Stefan Förster * Stefan Plewako * Stefaniu Criste * Stephan Richter * Stig Hackvan * Stonewall Ballard * Stuart Bishop * Students of HIT * Sven Anderson * Sylvain Langlade * Szabolcs Szigeti * Søren Bondrup * Tamito KAJIYAMA * Tanner Lovelace * Ted Cabeen * Terry Allen * Terry Grace * Terry Hardie * Thijs Kinkhorst * Tim Peters * Timothy O'Malley * Todd (Freedom Lover) * Todd Vierling * Todd Zullinger * Tollef Fog Heen * Tom G. Christensen * Tomasz Chmielewski * Toni Panadès * Tristan Roddis * Uros Kositer * Vadim Getmanshchuk * Valia V. Vaneeva * Vizi Szilard * Walter Hop * William Ahern * YASUDA Yukihiro * Yasuhito FUTATSUKI And everyone else on mailman-developers@python.org and mailman-users@python.org! Thank you, all. mailman-3.2.2/src/mailman/docs/NEWS.rst0000644000175000017500000023457013445505745020744 0ustar maxkingmaxking00000000000000======================= GNU Mailman 3 changes ======================= Copyright (C) 1998-2018 by the Free Software Foundation, Inc. 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA Here is a history of user visible changes to Mailman. 3.3.0 -- "Tom Sawyer" ===================== (20xx-xx-xx) 3.2.2 ===== (2019-03-23) REST ---- * Failed validation to new Mailing List creation and Domain creation no longer returns 500 errors. * Expose ``preferred_language`` for MailingList through REST API. * Self-link of banned emails will be url-encoded when the email is actually a regex, mainly to prevent misplaced `?` in the link. * REST API now accepts inputs in JSON format. (See !451) * REST API errors are now always formatted as JSON. (See !451) * Failed request validations now return more verbose errors, like ``Enum`` types return all acceptable values. (See !451) * REST API accepts request with content type None (See !479) Command line ------------ * The ``mailman import21`` command now displays import progress. (Closes #561) * An issue with ``mailman subcommand --help`` hanging has been worked around. (Closes #520) 3.2.1 ===== (2019-02-22) Command line ------------ * The ``mailman import21`` command properly converts all acceptable_aliases to regexps. (Closes #496) * The ``mailman import21`` command correctly converts autorespond_* settings. (Closes #505) * The various ``mailman`` commands now issue a help hint in case of error. Tests have been updated to recognize this. (Closes #519) Bugs ---- * Open non-regular log files (e.g. fifos) in write mode instead of append mode. (See !407) * Autoresponses to posts and -owner and -request messages now work. (Closes #504) * Message parts are now properly decoded when trying to remove an Approved: header. (Closes #518) * Outgoing SMTP connections are now closed following message delivery regardless of the max_sessions_per_connection setting. (Closes #529) * Messages with Subject: Re: only are now processed correctly. (Closes #545) LOCALIZATION ------------ * Added German translations for mail templates. Thanks to the "Institut für Gebrauchsgrafik" for providing them. (Closes #534) REST ---- * Allow setting ``max_num_recipients`` for a mailing list. (Closes #508) * Add a per-list visibility option for Members. (Closes #108) Other ----- * Email commands are now case insensitive. (Closes #353) * Implicit commands sent to -join, -leave and -confirm addresses no longer generate a Results email. (Closes #516) * An ``explicit_header_only`` option has been added to Reply-To: munging to avoid putting the list address in Cc:. (Closes #531) * The default list welcome message has been cleaned up a bit. (Closes #530) * A substitution variable ``user_name_or_email`` has been added to ``list:member:regular:header`` and ``list:member:regular:footer`` templates when personalized deliveries are enabled. (Closes #533) * The template search algorithm was fixed to look for in-tree templates for each language in the search list. (Closes #535) * Set default subscription policy to "confirm then moderate" for private mailing list style. (Closes #536) 3.2.0 -- "La Villa Strangiato" ============================== (2018-07-10) Bugs ---- * A missing html_to_plain_text_command is now properly detected and logged. (Closes #345) * Syntactically invalid sender addresses are now ignored. (Closes #229) * An AttributeError: 'str' object has no attribute 'decode' exception in subject prefixing is fixed. (Closes #359) * Messages with no syntactically valid senders are now automatically discarded. (Closes #369) * Various message holds and rejects that gave 'N/A' as a reason now give an appropriate reason. (Closes #368) * Bounce messages are now composed for proper translations. * Messages with badly encoded ``Subject`` headers no longer crash the message moderation REST API. (Closes #383) * Generated ``regexp`` tables for Postfix now account for possible ``+extra`` additions to the ``-bounces`` and ``-confirm`` addresses. (Closes #401) * Address confirmation notices are now properly encoded in the character set of the list's preferred_language. (Closes #395 and #413) * Thanks to Jim Popovitch, certain failures in DNS lookups of DMARC policy will now result in mitigations being applied. (Closes #415) * Messages without a sender can no longer bypass the ``nonmember-moderation`` rule. (Closes #414) * Invalid regexps in header_matches rules are properly logged and can't be set via REST. (Closes #418) * A list whose name is one of the ``admin``, ``bounces``, ``confirm``, etc. subaddresses can now be posted to. (Closes #433) * The ``admin`` subaddress, a synonym for ``bounces`` and deprecated since Mailman 2.1, has been removed. (Closes #435) * Better support for changing the ``list_name`` property. (Closes #428) * Raw Message text is now coerced to pure ascii before sending, and https://bugs.python.org/issue27321 is now defended against by overriding Message.as_string() to cover more cases than before. (Closes #406) * The override of Message.as_string() has been extended to catch another observed exception. (Closes #441) * The email.message.UserNotification() constructor will now not fail if given a ``text`` argument containing a character not in the charset of the ``lang`` argument. (Closes #437) * A new SQLAlchemy column type ``SAUnicodeXL`` has been implemented to support large columns in MySQL and is used for the ``value`` column of the ``pendedkeyvalue`` table. (Closes #385) * Messages with ``Subject`` headers encoded in an unknown character set no longer throw ``LookupError`` in subject prefixing. (Closes #445) * A list's ``last_post_at`` is now properly updated. (Closes #453) * Fixed an AttributeError in mailman/handlers/decorate.py when member.subscriber is an ``IUser`` instance rather than an ``IAddress`` instance. (Closes #449) * Message decoration has been removed from the posting pipeline and added to bulk delivery to avoid duplicate decoration of personalized and VERPed messages. (Closes #450) * The ``mailman inject`` command now reads standard input as documented if there is no ``-f`` or ``--filename`` option. (Closes #459) * Mailman doesn't chmod existing configuration directories on startup (Closes #439) * Calling ``set`` on a template evicts the local cache if the template was cached before. (Closes #463) * The override of Message.as_string() has been extended to catch yet another observed exception. (Closes #470) * Fixed a typo in the help for ``mailman digests --periodic``. (Closes #472) * Character encoding errors in adding headers and footers to multipart messages are detected and ``replaced``. (Closes #409) * The help for ``mailman qfile`` has been fixed to reference ``mailman qfile`` rather than the non-existent ``bin/dumpdb``. (Closes #475) * The dmarc rule has been fixed to not throw a TypeError when msg.get('from') returns a header instance. (Closes #477) * ``mailman`` command without any sub-command now invokes ``mailman help`` by default. (Closes #479) * The ``alias_domain`` attribute can become a null string. ``mta/postfix.py`` has been fixed to generate proper mappings in this case. (Closes #485) * Fix a bug where duplicate address records can be created when a user adds an existing address with a different case. (Closes #476) * Several additional templates can now be set in the ``var/templates`` hierarchy. (Closes #486) Command line ------------ * Adopt the ``click`` package for command line parsing. This makes the command line more consistent and pluggable. Also, many CLIs that accepted a "fqdn list name" (i.e. the posting address of a mailing list), now also accept a ``List-ID``. Every attempt has been made to keep the CLI backward compatible, but there may be subtle differences. (Closes #346) * If no listname is given, running ``mailman withlist -r`` must name a function taking no arguments. This can be used to introspect Mailman outside of the context of a mailing list. * Fix ``mailman withlist`` command parsing. (Closes #319) * Add a new option ``--periodic`` to ``mailman digests`` command. It sends out digests for only those Mailing Lists that have ``digest_send_periodic`` option set to ``True``. (Closes #384) * The ``mailman import21`` command now imports DMARC mitigations. * The ``mailman import21`` command no longer creates templates with non-standard names, (Closes #482) Configuration ------------- * The ``[mailman]pre_hook`` and ``[mailman]post_hook`` variables are deprecated. They can still be specified but they will not be run. * The ``[paths.*]ext_dir`` variable has been removed. * A new logger has been added called ``logging.plugins``. * The ``[styles]paths`` variable has been removed; you can now specify additional styles using the new plugin architecture. * Mailman now also searches at ``/etc/mailman3/mailman.cfg`` for the configuration file. Database -------- * The fix for #313 which ported an upstream SQLAlchemy fix to Mailman has been refactored now that SQLAlchemy 1.2 has been released. Interfaces ---------- * Broaden the semantics for ``IListManager.get()``. This API now accepts both ``List-ID``s and fully qualified list names, since that's the most common use case. There's now a separate ``.get_by_fqdn()`` which only accepts the latter and mirrors the already existing ``.get_by_list_id()``. * A new template ``list:user:notice:rejected`` has been added for customizing the bounce message rejection notice. * The ``acceptable_alias`` and ``require_explicit_destination`` attributes have been added to the ``IMailingList`` definition. (Closes #483) Other ----- * Add a new plugin architecture, which allows third parties to add initialization hooks, REST endpoints, and additional components. Given by Jan Jancar. * Drop support for Python 3.4. (Closes #373) * Bump minimum requirements for aiosmtpd (>= 1.1) and flufl.lock (>= 3.1). * Add '.pc' (patch directory) to list of ignored patterns when building the documentation with Sphinx. * Domain objects now have an ``alias_domain`` attribute which is normally ``None``, but which can be set to an alternate domain. This will enable the Postfix MTA to generate ``virtual_alias_maps`` which will map list addresses in the domain to addresses in the alternate domain, and will modify the other generated mappings to use the ``alias_domain``. The ``lmtp`` runner will also accept the alias_domain as an alias for the ``email_host``. * ``ICacheManager`` now allows evicting a single file from cache. REST ---- * Allow a mailing list's acceptable aliases to be cleared by calling ``DELETE`` on the list's ``config/acceptable_aliases`` resource. (Closes #394) * Allow setting ``max_message_size`` for a mailing list. (Closes #417) * Added new attribute in ``lists/styles`` resource ``styles`` that contains the list of all the styles in Core along with their description. ``style_names`` attribute is now deprecated and will be removed in future versions. * Add ``display_name`` for ``member`` in order to expose ``member.display_name``. (Closes #398) * Expose ``MailingList.respond_to_post_request`` through REST API. (Closes #420) * Add a new API ``lists/find`` which returns all the lists related to a subscriber. It optionally allows filtering based on a role. (See !388) * ``IAcceptableAliasSet`` resource now interprets an empty string ('') as an empty list ([]). This can be used to clear the list of acceptable aliases of a MailingList in a PATCH or PUT request, without having to use a DELETE request. * Expose ``MailingList.require_explicit_destination`` through REST API. (Closes #484) 3.1.0 -- "Between The Wheels" ============================= (2017-05-25) Bugs ---- * When the mailing list's ``admin_notify_mchanges`` is True, the list owners now get the subscription notification. (Closes: #1) * Fix the traceback that occurred when trying to convert a ``text/html`` subpart to plaintext via the ``mimedel`` handler. Now, a configuration variable ``[mailman]html_to_plain_text_command`` in the ``mailman.cfg`` file defines the command to use. It defaults to ``lynx``. (Closes: #109) * Confirmation messages should not be ``Precedence: bulk``. (Closes #75) * Fix constraint violations on mailing list deletes affecting PostgreSQL. Given by Abhilash Raj. (Closes #115) * ``mailman`` command with no subcommand now prints the help text. Given by Abhilash Raj. (Closes #137) * The MHonArc archiver must set stdin=PIPE when calling the subprocess. Given by Walter Doekes. * For now, treat ``DeliveryMode.summary_digests`` the same as ``.mime_digests``. (Closes #141). Also, don't enqueue a particular digest if there are no recipients for that digest. * For Python versions earlier than 3.5, use a compatibility layer for a backported smtpd module which can accept non-UTF-8 data. (Closes #140) * Bulk emails are now decorated with headers and footers. Given by Aurélien Bompard. (Closes #145) * Core no longer depends on the standalone ``mock`` module. (Closes: #146) * The logging of moderation reasons has been fixed. Given by Aurélien Bompard. * Collapse multiple ``Re:`` in Subject headers. Given by Mark Sapiro. (Closes: #147) * Added Trove classifiers to setup.py. (Closes: #152) * Fix the processing of subscription confirmation messages when the mailing list is set to confirm-then-moderate. (Closes #114) * Fix ``UnicodeEncodeError`` in the hold chain when sending the authorization email to the mailing list moderators. (Closes: #144) * Fix traceback in approved handler when the moderator password is None. Given by Aurélien Bompard. * Fix IntegrityErrors raised under PostreSQL when deleting users and addresses. Given by Aurélien Bompard. * Allow mailing lists to have localhost names with a suffix matching the subcommand extensions. Given by Aurélien Bompard. (Closes: #168) * Don't traceback if a nonexistent message-id is deleted from the message store. Given by Aurélien Bompard, tweaked by Barry Warsaw. (Closes: #167) * Fix a bug in ``SubscriptionService.find_members()`` when searching for a subscribed address that is not linked to a user. Given by Aurélien Bompard. * Fix a REST server crash when trying to subscribe a user without a preferred address. (Closes #185) * Fix membership query when multiple users are subscribed to a mailing list. Reported by Darrell Kresge. (Closes: #190) * Prevent moderation of messages held for a different list. (Closes: #161) * When approving a subscription request via the REST API, for a user who is already a member, return an HTTP 409 Conflict code instead of the previous server traceback (and resulting HTTP 500 code). (Closes: #193) * In decoration URIs (e.g. ``IMailingList.header_uri`` and ``.footer_uri``) you should now use the mailing list's List-ID instead of the fqdn-listname. The latter is deprecated. (Closes #196) * Trying to subscribe an address as a list owner (or moderator or nonmember) which is already subscribed with that role produces a server error. Originally given by Anirudh Dahiya. (Closes #198) * Cross-posting messages held on both lists no longer fails. (Closes #176) * Don't let unknown charsets crash the "approved" rule. Given by Aurélien Bompard. (Closes #203) * Don't let crashes in IArchiver plugins break handlers or runners. (Closes #208) * Fix "None" as display name in welcome message. Given by Aditya Divekar. (Closes #194) * Fix ``mailman shell`` processing of ``$PYTHONSTARTUP``. (Closes #224) * Fix query bug for ``SubscriptionService.find_members()`` leading to the incorrect number of members being returned. Given by Aurélien Bompard. (Closes #227) * Fix header match rule suffix inflation. Given by Aurélien Bompard. (Closes #226) * MIME digests now put the individual message/rfc822 messages inside a multipart/digest subpart. (Closes #234) * Nonmember subscriptions are removed when one of the addresses controlled by a user is subscribed as a member. Given by Aditya Divekar. (Closes #237) * Email address validation is now more compliant with RFC 5321. (Closes #266) * A mailing list's ``description`` must not contain newlines. Given by Aurélien Bompard. (Closes: #273) * Allow MailingList.info to be set using the REST API. Given by Aurélien Bompard. * Extend header filters to also check sub-part headers. (Closes #280) * Allow REST API to PUT and PATCH domain attributes. Allows Postorius domain edit to work. (Closes: #290) * Prevent posting from banned addresses. Given by Aurélien Bompard. (Closes: #283) * Remove the digest mbox files after the digests are sent. Given by Aurélien Bompard. (Closes: #259) * Transmit the moderation reason and expose it in the REST API as the ``reason`` attribute. Given by Aurélien Bompard. * Don't return a 500 error from the REST API when trying to handle a held message with defective content. Given by Abhilash Raj. (Closes: #256) * Delete subscription requests when a mailing list is deleted. Given by Abhilash Raj. (Closes: #214) * Messages were shunted when non-ASCII characters appeared in a mailing list's description. Given by Mark Sapiro. (Closes: #215) * Fix confirmation of unsubscription requests. (Closes: #294) * Fix ``mailman stop`` not stopping some runners due to PEP 475 interaction. (Closes: #255) * Update documentation links for ``config.cfg`` settings. (Closes: #306) * Disallow problematic characters in listnames. (Closes: #311) * Forward port several content filtering fixes from the 2.1 branch. (Closes: #330, #331, #332 and #334) Configuration ------------- * Mailing lists can now have their own header matching rules, although site-defined rules still take precedence. Importing a Mailman 2.1 list with header matching rules defined will create them in Mailman 3, albeit with a few unsupported corner cases. Definition of new header matching rules is not yet exposed through the REST API. Given by Aurélien Bompard. * The default languages from Mailman 2.1 have been ported over. Given by Aurélien Bompard. * There is now a configuration setting to limit the characters that can be used in list names. Command line ------------ * ``mailman create `` will now create missing domains by default. The ``-d``/``--domain`` option is kept for backward compatibility, but now there is a ``-D``/``--no-domain`` option to prevent missing domains from being create, forcing an error in those cases. Given by Gurkirpal Singh. (Closes #39) * ``mailman`` subcommands now properly commit any outstanding transactions. (Closes #223) * ``mailman digests`` has grown ``--verbose`` and ``-dry-run`` options. * ``mailman shell`` now supports readline history if you set the ``[shell]history_file`` variable in mailman.cfg. Also, many useful names are pre-populated in the namespace of the shell. (Closes: #228) Database -------- * MySQL is now an officially supported database. Given by Abhilash Raj. * Fix a problem with tracebacks when a PostgreSQL database is power cycled while Mailman is still running. This ports an upstream SQLAlchemy fix to Mailman in lieu of a future SQLAlchemy 1.2 release. (Closes: #313) Interfaces ---------- * Implement reasons for why a message is being held for moderator approval. Given by Aurélien Bompard, tweaked by Barry Warsaw. * The default ``postauth.txt`` and ``postheld.txt`` templates now no longer include the inaccurate admindb and confirmation urls. * Messages now include a ``Message-ID-Hash`` as the replacement for ``X-Message-ID-Hash`` although the latter is still included for backward compatibility. Also be sure that all places which add the header use the same algorithm. (Closes #118) * ``IMessageStore.delete_message()`` no longer raises a ``LookupError`` when you attempt to delete a nonexistent message from the message store. * ``ISubscriptionService.find_members()`` accepts asterisks as wildcards in the ``subscriber`` argument string. Given by Aurélien Bompard. * ``ISubscriptionService`` now supports mass unsubscribes. Given by Harshit Bansal. Message handling ---------------- * New DMARC mitigations have been added. Given by Mark Sapiro. (Closes #247) * New placeholders have been added for message headers and footers. You can use a placeholder of the format ``$_url`` to insert the permalink to the message in the named archiver, for any archiver enabled for the mailing list. Given by Abhilash Raj. * The default posting chain has been modified so that the header-match chain and nonmember-moderation rule are processed before "hold" rules are processed. This allows for better anti-spam defenses and rejecting non-member posts instead of always holding them for moderator review. Given by Aurélien Bompard. (Closes #163) * Bounces can now contain rejection messages. Given by Aurélien Bompard. * The ``moderation_action`` for members and nonmember can now be ``None`` which signals falling back to the appropriate list default action, e.g. ``default_member_action`` and ``default_nonmember_action``. Given by Aurélien Bompard. (Closes #189) * Ensure that postings from alternative emails aren't held for moderator approval. For example, if a user is subscribed with one email but posts with a second email that they control, the message should be processed as a posting from a member. Given by Aditya Divekar. (Closes #222) * The default message footer has been improved to include a way to unsubscribe via the ``-leave`` address. Given by Francesco Ariis. REST ---- * REST API version 3.1 introduced. Mostly backward compatible with version 3.0 except that UUIDs are represented as hex strings instead of 128-bit integers, since the latter are not compatible with all versions of JavaScript. (Closes #121) * REST clients must minimally support HTTP/1.1. (Closes #288) * Experimental Gunicorn support. See ``contrib/gunicorn.py`` docstring for details. With assistance from Eric Searcy. (Closes #287) * The new template system is introduced for API 3.1. See ``src/mailman/rest/docs/templates.rst`` for details. (Closes #249) * When creating a user via REST using an address that already exists, but isn't linked, the address is linked to the new user. Given by Aurélien Bompard. * The REST API incorrectly parsed ``is_server_owner`` values when given explicitly in the POST that creates a user. (Closes #136) * A new top-level resource ``/owners`` can be used to get the list of server owners as ``IUser`` s. (Closes #135) * By POSTing to a user resource with an existing unlinked address, you can link the address to the user. Given by Abhilash Raj. * Fix pagination values ``start`` and ``total_size`` in the REST API. Given by Aurélien Bompard. (Closes: #154) * JSON representations for held message now include a ``self_link``. * When ``[devmode]enabled`` is set, the JSON output is sorted. Given by Aurélien Bompard. * A member's moderation action can be changed via the REST API. Given by Aurélien Bompard. * Fixed a number of corner cases for the return codes when PUTing or PATCHing list configuration variables. (Closes: #182) * Expose ``digest_send_periodic``, ``digest_volume_frequency``, and ``digests_enabled`` (renamed from ``digestable``) to the REST API. (Closes: #159) * Expose the "bump digest" and "send digest" functionality though the REST API via the ``/lists//digest`` end-point. GETting this resource returns the ``next_digest_number`` and ``volume`` as the same values accessible through the list's configuraiton resource. POSTing to the resource with either ``send=True``, ``bump=True``, or both invokes the given action. * Global and list-centric bans can now be managed through the REST API. Given by Aurélien Bompard. * ``/members/find`` accepts GET query parameters in addition to POST arguments. Given by Aurélien Bompard. * Header match rules for individual mailing lists are now exposed in the REST API. Given by Aurélien Bompard. (Closes: #192) * Expose ``goodbye_message_uri`` in the REST API. Given by Harshit Bansal. * New subscription requests are rejected if there is already one pending. With thanks to Anirudh Dahiya. (Closes #199) * Expose the system pipelines and chains via ``/system/pipelines`` and ``/system/chains`` respectively. Given by Simon Hanna. (Closes #66) * Support mass unsubscription of members via ``DELETE`` on the ``/lists//roster/member`` resource. Given by Harshit Bansal. (Closes #171) * It is now possible to merge users when creating them via REST. When you POST to ``/users/
/addresses`` and the address given in the ``email`` parameter already exists, instead of getting a 400 error, if you set ``absorb_existing=True`` in the POST data, the existing user will be merged into the newly created on. Given by Aurélien Bompard. * Port to Falcon 1.0 (Closes #20) * A member's ``moderation_action`` can be reset, allowing fallback to the list's ``default_member_action`` by setting the attribute to the empty string in the REST API. Given by Aurélien Bompard. * A list's ``moderator_password`` can be set via the REST API. Given by Andrew Breksa. (Closes #207) * The ban manager now returns a pageable, sorted sequence. Given by Amit and Aurélien Bompard. (Closes #284) * Query parameters now allow you to filter mailing lists by the ``advertised`` boolean parameter. Given by Aurélien Bompard. * Only the system-enabled archivers are returned in the REST API. Given by Aurélien Bompard. * **Backward incompatibility: mild** Held message resources now have an ``original_subject`` key which is the raw value of the ``Subject:`` header (i.e. without any RFC 2047 decoding). The ``subject`` key is RFC 2047 decoded. Given by Simon Hanna. (Closes #219) Other ----- * Add official support for Python 3.5 and 3.6. (Closes #295) * A handful of unused legacy exceptions have been removed. The redundant ``MailmanException`` has been removed; use ``MailmanError`` everywhere. * Drop the use of the ``lazr.smtptest`` library, which is based on the asynchat/asyncore-based smtpd.py stdlib module. Instead, use the asyncio-based `aiosmtpd `_ package. * Improvements in importing Mailman 2.1 lists, given by Aurélien Bompard. * The ``prototype`` archiver is not web accessible so it does not have a ``list_url`` or permalink. Given by Aurélien Bompard. * Large performance improvement in ``SubscriptionService.find_members()``. Given by Aurélien Bompard. * Rework the digest machinery, and add a new ``digests`` subcommand, which can be used from the command line or cron to immediately send out any partially collected digests, or bump the digest and volume numbers. * The mailing list "data directory" has been renamed. Instead of using the fqdn listname, the subdirectory inside ``[paths]list_data_dir`` now uses the List-ID. * The ``mailman members`` command can now be used to display members based on subscription roles. Also, the positional "list" argument can now accept list names or list-ids. * Unsubscriptions can now be confirmed and/or moderated. (Closes #213) 3.0.0 -- "Show Don't Tell" ========================== (2015-04-28) Architecture ------------ * Domains now have a list of owners, which are ``IUser`` objects, instead of the single ``contact_address`` they used to have. ``IUser`` objects now also have a ``is_server_owner`` flag (defaulting to False) to indicate whether they have superuser privileges. Give by Abhliash Raj, with fixes and refinements by Barry Warsaw. (LP: #1423756) * Mailing list subscription policy work flow has been completely rewritten. It now properly supports email verification and subscription confirmation by the user, and approval by the moderator using unique tokens. ``IMailingList`` objects now have a ``subscription_policy`` attribute. (LP: #1095552) * Port the REST machinery to Falcon 0.3. (LP: #1446881) Bugs ---- * Fix calculation of default configuration file to use when the ``$var_dir`` is created by ``mailman start``. (LP: #1411435) * When creating a user with an email address, do not create the user record if the email address already exists. Given by Andrew Stuart. (LP: #1418280) * When deleting a user via REST, make sure all linked addresses are deleted. Found by Andrew Stuart. (LP: #1419519) * When trying to subscribe an address to a mailing list through the REST API where a case-differing version of the address is already subscribed, return a 409 error instead of a 500 error. Found by Ankush Sharma. (LP: #1425359) * ``mailman lists --domain`` was not properly handling its arguments. Given by Manish Gill. (LP: #1166911) * When deleting a user object, make sure their preferences are also deleted. Given by Abhishek. (LP: #1418276) * Be sure a mailing list's acceptable aliases are deleted when the mailing list itself is deleted. (LP: #1432239) * The built-in example ``IArchiver`` implementations now explicitly return None. (LP: #1203359) * The test suite now runs successfully again with PostgreSQL. Given by Aurélien Bompard. (LP: #1435941) Configuration ------------- * When specifying a file system path in the [paths.*] section, $cfg_file can be used to expand into the path of the ``-C`` option if given. In the default ``[paths.dev]`` section, ``$var_dir`` is now specified relative to ``$cfg_file`` so that it won't accidentally be relative to the current working directory, if ``-C`` is given. * ``$cwd`` is now an additional substitution variable for the ``mailman.cfg`` file's ``[paths.*]`` sections. A new ``[paths.here]`` section is added, which puts the ``var_dir`` in ``$cwd``. It is made the default layout. Documentation ------------- * Improve the documentation describing how to run Alembic to add new schema migrations. Given by Abhilash Raj. REST ---- * **Backward incompatible change**: The JSON representation for pending mailing list subscription hold now no longer includes the ``password`` key. Also, the ``address`` key has been renamed ``email`` for consistent terminology and other usage. * You can now view the contents of, inject messages into, and delete messages from the various queue directories via the ``/queues`` resource. * You can now DELETE an address. If the address is linked to a user, the user is not delete, it is just unlinked. * A new API is provided to support non-production testing infrastructures, allowing a client to cull all orphaned UIDs via ``DELETE`` on ``/reserved/uids/orphans``. Note that *no guarantees* of API stability will ever be made for resources under ``reserved``. (LP: #1420083) * Domains can now optionally be created with owners; domain owners can be added after the fact; domain owners can be deleted. Also, users now have an ``is_server_owner`` flag as part of their representation, which defaults to False, and can be PUT and PATCH'd. Given by Abhilash Raj, with fixes and refinements by Barry Warsaw. (LP: #1423756) 3.0 beta 5 -- "Carve Away The Stone" ==================================== (2014-12-29) Bugs ---- * Fixed Unicode errors in the digest runner and when sending messages to the site owner as a fallback. Given by Aurélien Bompard. (LP: #1130957). * Fixed Unicode errors when a message being added to the digest has non-ascii characters in its payload, but no Content-Type header defining a charset. Given by Aurélien Bompard. (LP: #1170347) * Fixed messages without a `text/plain` part crashing the `Approved` rule. Given by Aurélien Bompard. (LP: #1158721) * Fixed getting non-ASCII filenames from RFC 2231 i18n'd messages. Given by Aurélien Bompard. (LP: #1060951) * Fixed `AttributeError` on MIME digest messages. Given by Aurélien Bompard. (LP: #1130696) Commands -------- * The `mailman conf` command no longer takes the `-t/--sort` option; the output is always sorted. Configuration ------------- * The ``[database]migrations_path`` setting is removed. Database -------- * The ORM layer, previously implemented with Storm, has been replaced by SQLAlchemy, thanks to the fantastic work by Abhilash Raj and Aurélien Bompard. Alembic is now used for all database schema migrations. * The new logger `mailman.database` logs any errors at the database layer. Development ----------- * Python 3.4 is now the minimum requirement. * You no longer have to create a virtual environment separately when running the test suite. Just use `tox`. * You no longer have to edit `src/mailman/testing/testing.cfg` to run the test suite against PostgreSQL. See `src/mailman/docs/START.rst` for details. Interfaces ---------- * The RFC 2369 headers added to outgoing messages are now added in sorted order. * Several changes to the internal API: - `IListManager.mailing_lists` is guaranteed to be sorted in List-ID order. - `IDomains.mailing_lists` is guaranteed to be sorted in List-ID order. - Iteration over domains via the `IDomainManager` is guaranteed to be sorted by `IDomain.mail_host` order. - `ITemporaryDatabase` interface and all implementations are removed. REST ---- * The Falcon Framework has replaced restish as the REST layer. This is an internal change only. * The JSON representation `http_etag` key uses an algorithm that is insensitive to Python's dictionary sort order. * The address resource now has an additional '/user' sub-resource which can be used to GET the address's linked user if there is one. This sub-resource also supports POST to link an unlinked address (with an optional 'auto_create' flag), and PUT to link the address to a different user. It also supports DELETE to unlink the address. (LP: #1312884) Given by Aurélien Bompard based on work by Nicolas Karageuzian. * The ``/3.0/system`` path is deprecated; use ``/3.0/system/versions`` to get the system version information. * You can access the system configuration via the resource path ``/3.0/system/configuration/
``. This returns a dictionary with the keys being the section's variables and the values being their value from ``mailman.cfg`` as verbatim strings. You can get a list of all section names via ``/3.0/system/configuration`` which returns a dictionary containing the ``http_etag`` and the section names as a sorted list under the ``sections`` key. The system configuration resource is read-only. * Member resource JSON now include the ``member_id`` as a separate key. 3.0 beta 4 -- "Time and Motion" =============================== (2014-04-22) Development ----------- * Mailman 3 no longer uses ``zc.buildout`` and tests are now run by the ``nose2`` test runner. See ``src/mailman/docs/START.rst`` for details on how to build Mailman and run the test suite. Also, use ``-P`` to select a test pattern and ``-E`` to enable stderr debugging in runners. * Use the ``enum34`` package instead of ``flufl.enum``. * Use ``setuptools`` instead of ``distribute``, since the latter is defunct. REST ---- * Add ``reply_to_address`` and ``first_strip_reply_to`` as writable attributes of a mailing list's configuration. (LP: #1157881) * Support pagination of some large collections (lists, users, members). [Florian Fuchs] (LP: #1156529) * Expose ``hide_address`` to the ``.../preferences`` REST API. [Sneha Priscilla.] (LP: #1203519) * Mailing lists can now individually enable or disable any archiver available site-wide. [Joanna Skrzeszewska] (LP: #1158040) * Addresses can be added to existing users, including display names, via the REST API. [Florian Fuchs] * Fixed a crash in the REST server when searching for nonmembers via ``/find`` which we've never seen before, because those members only have an address record, not a user record. This requires a small change in the API where the JSON response's ``address`` key now contains the URL to the address resource, the new ``email`` key contains the email address as a string, and the ``user`` key is optional. Commands -------- * `mailman conf` now has a `-t/--sort` flag which sorts the output by section and then key. [Karl-Aksel Puulmann and David Soto] (LP: 1162492) * Greatly improve the fidelity of the Mailman 2.1 list importer functionality (i.e. ``mailman import21``). [Aurélien Bompard]. Configuration ------------- * Add support for the Exim 4 MTA. [Stephen Turnbull] * When creating the initial file system layout in ``var``, e.g. via ``bin/mailman info``, add an ``var/etc/mailman.cfg`` file if one does not already exist. Also, when initializing the system, look for that file as the configuration file, just after ``./mailman.cfg`` and before ``~/.mailman.cfg``. (LP: #1157861) Database -------- * The `bounceevent` table now uses list-ids to cross-reference the mailing list, to match other tables. Similarly for the `IBounceEvent` interface. * Added a `listarchiver` table to support list-specific archivers. Bugs ---- * Non-queue runners should not create ``var/queue`` subdirectories. [Sandesh Kumar Agrawal] (LP: #1095422) * Creation of lists with upper case names should be coerced to lower case. (LP: #1117176) * Fix REST server crash on `mailman reopen` due to no interception of signals. (LP: #1184376) * Add `subject_prefix` to the `IMailingList` interface, and clarify the docstring for `display_name`. (LP: #1181498) * Fix importation from MM2.1 to MM3 of the archive policy. [Aurélien Bompard] (LP: #1227658) * Fix non-member moderation rule to prefer a member sender if both members and non-members are in the message's sender list. [Aurélien Bompard] (LP: #1291452) * Fix IntegrityError (against PostgreSQL) when deleting a list with content filters. [Aurélien Bompard] (LP: #1117174) * Fix test isolation bug in ``languages.rst``. [Piotr Kasprzyk] (LP: #1308769) 3.0 beta 3 -- "Here Again" ========================== (2012-12-31) Compatibility ------------- * Python 2.7 is now required. Python 2.6 is no longer officially supported. The code base is now also `python2.7 -3` clean, although there are still some warnings in 3rd party dependencies. (LP: #1073506) REST ---- * **API change**: The JSON representation for held messages no longer includes the `data` key. The values in this dictionary are flatted into the top-level JSON representation. The `key` key is remove since it's redundant. Use `message_id` for held messages, and `address` for held subscriptions/unsubscriptions. The following `_mod_*` keys are inserted without the `_mod_` prefix: - `_mod_subject` -> `subject` - `_mod_hold_date` -> `hold_date` - `_mod_reason` -> `reason` - `_mod_sender` -> `sender` - `_mod_message_id` -> `message_id` * List styles are supported through the REST API. Get the list of available styles (by name) via `.../lists/styles`. Create a list in a specific style by using POST data `style_name=