.. Reminder for header structure: Parts (H1) : #################### with overline Chapters (H2) : ******************** with overline Sections (H3) : ==================== Subsections (H4) : -------------------- Subsubsections (H5) : ^^^^^^^^^^^^^^^^^^^^ Paragraphs (H6) : """"""""""""""""""""" .. meta:: :description: Creating WAPT packages :keywords: working, WAPT, personalizing, creating packages, documentation .. |vscode| image:: wapt-resources/icon_visual-studio-text-editor.png :alt: Visual Studio text editor .. |vscodium| image:: wapt-resources/icon_vscodium-text-editor.png :alt: VSCodium text editor .. |vim| image:: wapt-resources/icon_vim-text-editor.png :alt: Vim text editor .. |pyscripter| image:: wapt-resources/icon_pyscripter-text-editor.png :alt: Pyscripter text editor .. |pycharm| image:: wapt-resources/icon_pycharm-text-editor.png :alt: Pycharm text editor .. |notepad++| image:: wapt-resources/icon_notepadpluplus-text-editor.png :alt: Notepad ++ text editor .. |nano| image:: wapt-resources/icon_nano-text-editor.png :alt: Nano text editor .. role:: green .. role:: orange .. role:: red .. _creating_WAPT_packages: #################################### Getting start creating WAPT packages #################################### ***************************************************** Setting up your WAPT development and test environment ***************************************************** Prerequisites ============= .. attention:: * It is **required** to be a :term:`Local Administrator` of the host to use WAPT's integrated environment for developing WAPT packages. * We advise you to create/ edit packages in a fully controlled environment that is safe and *disposable*. * The usage of a disposable virtual host (like Virtualbox) is recommended. * Import the *tis-waptdev* package in your local repository and install it on your development computer. Recommendations regarding the test environment ============================================== The recommended method to correctly test your WAPT packages is to use representative sample of hosts in your inventory. So the more heterogeneous your installed base of hosts, the larger your sample should be. This method consists of testing the installation of the package on as many platforms and configurations as possible, so to improve its reliability, before the WAPT package is transferred to production repositories. To explain how package maturities work, you can refer to this video for a detailed overview. .. youtube:: V-XFWzDkIUE :width: 50% :align: center Testing method ============== Operating systems and architectures ----------------------------------- | *Desktop Operating Systems* .. list-table:: :widths: 30, 30, 30 :align: left * - Windows XP - Windows 7 - Windows 10 * - Windows 11 - Redhat and derivatives - Debian and derivatives *Server Operating Systems* .. list-table:: :widths: 30, 30, 30 :align: left * - Windows Server 2008 R2 - Windows Server 2012 and R2 - Windows Server 2019 *Architectures* .. list-table:: :widths: 30, 30 :align: left * - x86 - x64 *Deployment Environments* .. list-table:: :widths: 30, 30 :align: left * - Physical and virtual hosts - Laptops When possible, RC and Beta version of Operating Systems should be tested. State of Windows Updates ------------------------ * **Microsoft Windows host without any Windows update installed**: the objective is to detect Windows updates that are required for the software to work properly and to adapt the WAPT package accordingly; * **Microsoft Windows host with all the latest Windows updates**: the objective is to detect Windows updates that break the package and to adapt the WAPT package accordingly; State of software installations ------------------------------- * **Hosts with many installed packages**: the objective is to detect a possible dependency with an existing application; * **Hosts with many installed packages**: the objective is to detect a possible conflict with an existing application; * **Install older versions of the software**: it is possible that the software installer does not support uninstalling a previous version of the software, in this case, the WAPT package will have to remove older versions of the software before installing the new version; .. _create_package_from_console: ************************************************************* Principles of creating package template from the WAPT Console ************************************************************* .. attention:: To create WAPT packages directly from the WAPT Console, it is necessary to have installed the WAPT development environment **tis-pyscripter3** or **tis-vscode**. It is NOT recommended to use **tis-pyscripter4**. We recommand you to download the **tis-waptdev** package instead and install it on your computer on which you will create WAPT packages. If you do not remember how to download a package from the Tranquil IT store, please see :ref:`how to download package in your private repository `. If you do not remember how to install a package, please see :ref:`how to install a package on a host `. Creating a package template from the WAPT Console ================================================= We use the 7-zip MSI setup downloaded from the 7-zip official website. * `Download tis-7zip for x64 `_. * Create a WAPT package Template from the installer. In the WAPT Console, click on :guilabel:`Tools` --> :guilabel:`Make package from setup from file`: .. figure:: wapt-resources/wapt_console_make-package-template_menu-option.png :scale: 75% :align: center :alt: Menu option for creating a WAPT package template in the WAPT Console Menu option for creating a WAPT package template in the WAPT Console Select the downloaded MSI setup file (:guilabel:`Windows installer(.msi/.msix/.exe)`) and **fill in the required fields**. Verify that the **package name does not contains any version number**. .. figure:: wapt-resources/wapt_console_package-wizard1_dialog-box.png :scale: 75% :align: center :alt: Dialog box requesting information when creating the WAPT package in the WAPT Console Dialog box requesting information when creating the WAPT package in the WAPT Console * **Two solutions are available:** * Click on :guilabel:`Edit manually` (recommended) to verify the WAPT package and customize it to your Organization's specific needs. * Click on :guilabel:`Build and upload` to directly build and upload the package into your private repository. .. Warning:: The button :guilabel:`Build and upload` directly uploads the package into the private repository **without testing**. This method works relatively well with MSI installers because their installation is more standardized. However, the **first method** that consists of first testing locally the package before uploading **is the recommended method**. .. note:: If you prefer to use the command line, :ref:`you can use this method `. Customizing the WAPT package before uploading it to the repository ================================================================== Before uploading a package to your WAPT repository, you may choose to customize its behavior to your Organization's needs by editing it with :program:`PyScripter`. When creating the package template, click on :guilabel:`Edit manually`. .. figure:: wapt-resources/wapt_console_package-wizard_make-and-edit-button_dialog-box.png :scale: 75% :align: center :alt: Dialog box highlighting the "Edit manuall" button when creating the WAPT package in the WAPT Console "Edit manually" button in dialog box A new dialog box will appear, advising that the WAPT package has been downloaded to the WAPT repository. .. figure:: wapt-resources/wapt_console_package-wizard-downloaded_ok-message-window.png :align: center :alt: Message window showing in the WAPT Console that the WAPT package has been downloaded into the WAPT repository Message windows for information The :program:`PyScripter` IDE launches automatically to allow you to edit files in the WAPT package. .. figure:: wapt-resources/windows_pyscripter_package-template-opened_container-window.png :scale: 50% :align: center :alt: PyScripter - Customizing a WAPT package within PyScripter PyScripter - Customizing a WAPT package within PyScripter Presentation of PyScripter ========================== PyScripter project explorer --------------------------- This is the initial view of the Pyscripter explorer when you open a WAPT Package. .. figure:: wapt-resources/Windows_pyscripter-explorer.png :align: center :alt: PyScripter - Navigating a project within the PyScrypter file explorer PyScripter - Navigating a project within the PyScrypter file explorer The PyScripter project explorer lists the different files that you might need, notably the :file:`control` file and the :file:`setup.py` file. Editor panel ------------ :orange:`The editing panel in orange` in the image above. :orange:`The edition panel` in :program:`PyScripter` allows to edit the :file:`setup.py` file and the :file:`control` file. Run Configurations ------------------ In the image above, the :green:`green box highlights` the pre-installed configurations. The :command:`Run` option in the project explorer of :program:`PyScripter` will allow you to launch actions on the packages that you are editing. These functions are explained in the table below. .. list-table:: List of functions for create package. :header-rows: 1 :widths: auto * - Function - Description * - install - Used to install the package on the system of our test machine. * - remove - Used to uninstall the package from the system of the test machine. * - uninstall - Enables you to run a custom uninstall of the test machine package. * - session-setup - Allows the package to be installed via the user context of the test machine. * - audit - Enables the package to be audited. * - update-package-source - Enables automation of a software package binary. Python console -------------- .. figure:: wapt-resources/windows_pyscripter-project-zone-python-running_container-window.png :align: center :alt: PyScripter - Running the Python console from within Pyscripter PyScripter - Running the Python console from within Pyscripter This is the Python console visible in :program:`PyScripter`, it will allow you to display the python output when you execute :command:`Run` commands. You can also use it to test/ debug portions of your script :file:`setup.py`. To learn more about the composition of a WAPT package, visit the documentation on the :ref:`structure of a WAPT package `. Testing locally the installation of the WAPT package ---------------------------------------------------- You can then test the launch of an installation on your development station. .. image:: wapt-resources/windows_pyscripter_run-install_menu-item.png :align: center :alt: PyScripter - Running an install command from the PyScripter console The PyScripter console allows you to check whether the installation went well. Testing locally the uninstallation of the WAPT package ------------------------------------------------------ You can then test the uninstall script on your development station. .. image:: wapt-resources/windows_pyscripter_run-remove_menu-item.png :align: center :alt: PyScripter - Running a remove command from the PyScripter console The PyScripter console allows you to check whether the uninstallation went well. Exploring Package Files with the WAPT Console ============================================= To explore all the files of a WAPT package using the WAPT Console, you must enable the :guilabel:`Show Developer Features` option. To do this: * Go to the :kbd:`View` tab in the WAPT Console. * Select :guilabel:`Display Preferences` and activate the :guilabel:`Show Developer Features checkbox`. How to explore package files ? Navigate to :command:`WAPT Packages` section, select your package and do a right-click on :kbd:`Edit_package` Once enabled, you can access the file explorer for the package. .. figure:: wapt-resources/WAPT-Console_techview.png :scale: 50% :align: center :alt: WAPT Console techview WAPT Console techview ********************************* Creating your first WAPT Packages ********************************* Packaging .msi packages (example) ================================= For this example we will take :program:`tightvnc`. You can download it `here `_. Now, you can then generate your package template, please refer to the :ref:`documentation for creating packages from the WAPT Console `. Edit the :file:`control` file (:code:`architecture`, :code:`impacted_process`, :code:`target_os`, :code:`description`, :code:`maintainer` ...). For more information, visit the :ref:`documentation on the control file structure `. Your :program:`PyScripter` opens, go to your :file:`setup.py`: .. code-block:: python # -*- coding: utf-8 -*- from setuphelpers import * uninstallkey = [] def install(): print('installing tis-tightvnc') install_msi_if_needed('tightvnc-2.8.5-setup-64bit.msi') * The function will test whether a version of the software is already installed on the host using the *uninstall key*. * If the *uninstall key* is already present, the new version of the software will be installed only if the installed version is older. * After the installation, the function will finally test that the *uninstall key* is present and its version, to ascertain that all went well. .. list-table:: List of arguments available with *install_msi_if_needed* :header-rows: 1 :widths: 50, 50 :align: center * - Options (Default Option) - Description * - :code:`min_version` (default ``None``) - Defines the minimal version above which the software will update. * - :code:`killbefore` (default ``None``) - Lists the programs to kill before installing the package. * - :code:`accept_returncodes` (default ``[0,3010]``) - Defines the accepted codes other than 0 and 3010 returned by the function. * - :code:`timeout` (default ``300``) - Defines the maximum installation wait time (in seconds). * - :code:`properties` (default ``None``) - Defines the additional properties to pass as arguments to MSI setup file. * - :code:`get_version` (default ``None``) - Defines the value passed as parameter to control the version number instead of the value returned by the *installed_softwares* function. * - :code:`remove_old_version` (default ``False``) - Automatically removes an older version of a software whose *uninstall key* is identical. * - :code:`force` (default ``False``) - Forces the installation of the software even though the same *uninstall key* has been found. The :command:`wapt-get install_msi_if_needed` method searches for an *uninstall key* in the MSI file properties, it is not necessary to fill it manually in the :file:`setup.py` file. You also do not have to fill in :code:`killbefore` if the value specified in the :code:`impacted_process` field of the :file:`control` file is correct. .. note:: The :file:`setup.py` could have looked like this, but the method is less elegant because it does less checking. .. code-block:: python # -*- coding: utf-8 -*- from setuphelpers import * uninstallkey = ["{8B9896FC-B4F2-44CD-8B6E-78A0B1851B59}"] def install(): print('installing tis-tightvnc') run('msiexec /norestart /q /i "tightvnc-2.8.5-setup-64bit.msi"') Run the installation and see what happens when the software is already installed. .. code-block:: bash wapt-get -ldebug install C:\waptdev\tis-tightvnc-wapt Installing WAPT file C:\waptdev\tis-tightvnc-wapt MSI tightvnc-2.8.5-gpl-setup-64bit.msi already installed. Skipping msiexec Results: === install packages === C:\waptdev\tis-tightvnc-wapt | tis-tightvnc (2.8.5.0-1) Passing additional arguments ---------------------------- To pass additional arguments, store them in a *dict*. .. code-block:: python # -*- coding: utf-8 -*- from setuphelpers import * uninstallkey = [] properties = { 'SERVER_REGISTER_AS_SERVICE': 0, 'SERVER_ADD_FIREWALL_EXCEPTION': 0, 'ADDLOCAL': 'Server,Viewer' } def install(): print(u'Installation en cours de TightVNC') install_msi_if_needed('tightvnc-2.8.5-setup-64bit.msi', properties = properties ) .. note:: The :file:`setup.py` could have looked like this, but the method is less elegant because it does less checking. .. code-block:: python # -*- coding: utf-8 -*- from setuphelpers import * uninstallkey = ["{8B9896FC-B4F2-44CD-8B6E-78A0B1851B59}"] def install(): print('installing tis-tightvnc') run('msiexec /norestart /q /i "tightvnc-2.8.5-setup-64bit.msi" SERVER_REGISTER_AS_SERVICE=0 SERVER_ADD_FIREWALL_EXCEPTION=0') Video demonstration ------------------- .. youtube:: Z6wr6emPGCU :align: center :width: 50% .. _simple_exe_packaging: Packaging .exe packages (example) ================================= * Download the :mimetype:`.exe` installer from a reliable source. Download the installer in exe format Firefox ESR x64 on ``_. * Look up documentation relating to silent flags: * On the `Official Mozilla website `_. * Other methods for finding information on silent flags: * `WPKG packages repository `_; * `Chocolatey packages repository `_; * Search on the Internet with the search terms: *Firefox silent install*. * Then generate your package template, please refer to the :ref:`documentation for creating packages from the WAPT Console `. :program:`PyScripter` loads up and opens the :mimetype:`.exe` package project. .. figure:: wapt-resources/windows_pyscripter_firefox_esr_browser-window.png :scale: 50% :align: center :alt: PyScripter - Opening the FirefoxESR WAPT package PyScripter - Opening the FirefoxESR WAPT package * Edit the :file:`control` file (:code:`architecture`, :code:`impacted_process`, :code:`target_os`, :code:`description`, :code:`maintainer` ...). For more information, visit the :ref:`documentation on the control file structure `. * Check the :file:`control` file content. Mozilla Firefox-ESR does not comply to industry standards and returns an erroneous version number (it appears to be the installer packaging software version number). * Original :file:`control` file. .. literalinclude:: wapt-resources/package-exe-control_origin.txt :emphasize-lines: 2 * Modified :file:`control` file. .. literalinclude:: wapt-resources/package-exe-control_modified.txt :emphasize-lines: 2,6,7,8 A sub-version *-1* has been appended to the software version number; it is the packaging version of the WAPT package. It allows the Package Developer to release several WAPT package versions of the same software, very useful for very rapid and iterative development. Using *install_exe_if_needed* The function is slightly the same as that used with :mimetype:`.msi` installers, with some differences: * The function requires to pass the silent flag as an argument. * The function requires to pass the *uninstall key* as an argument. .. code-block:: python # -*- coding: utf-8 -*- from setuphelpers import * uninstallkey = [] def install(): print('installing tis-firefox-esr') install_exe_if_needed("Firefox Setup 45.5.0esr.exe",silentflags="-ms",key='',min_version="4.42.0.0") .. list-table:: List of arguments available with *install_exe_if_needed* :header-rows: 1 :widths: 50, 50 :align: center * - Options (Default Option) - Description * - :code:`silentflags` (default ``None``) - Defines the silent parameters to pass as arguments to the installer. * - :code:`key` (default ``None``) - Defines the software *uninstall key*. * - :code:`min_version` (default ``None``) - Defines the minimal version above which the software will update. * - :code:`killbefore` (default ``None``) - Lists the programs to kill before installing the package. * - :code:`accept_returncodes` (default ``[0,3010]``) - Defines the accepted codes other than 0 and 3010 returned by the function. * - :code:`timeout` (default ``300``) - Defines the maximum installation wait time (in seconds). * - :code:`get_version` (default ``None``) - Defines the value passed as parameter to control the version number instead of the value returned by the *installed_softwares* function. Example ``_. * - :code:`remove_old_version` (default ``False``) - Automatically removes an older version of a software whose *uninstall key* is identical. * - :code:`force` (default ``False``) - Forces the installation of the software even though the same *uninstall key* has been found. The package will then have this behavior: * Firefox will install only if Firefox is not yet installed or if the installed version of Firefox is less than 45.5.0, unless the :code:`--force` option is passed as argument when installing the package. * On installing, the running :program:`firefox.exe` processes will be killed (with the value indicated in :code:`impacted_process` of the :file:`control` file). * The function will add by itself the *uninstall key*, so leave the *uninstall key* argument empty. * When finishing the installation of the package, the function will check that the *uninstall key* is present on the host and that the version of Firefox is greater than 45.5.0; if this not the case, the package will be flagged as **ERROR**. Finding the uninstallation key ------------------------------ Unlike :mimetype:`.msi` files, the key to uninstall an :mimetype:`.exe` is not in the file properties. So you need to install the software first to know the uninstall key. Therefore you **MUST** start once the installation from :program:`PyScripter` with the :guilabel:`run configuration` and then :guilabel:`install`. .. image:: wapt-resources/windows_pyscripter_run-install_menu-item.png :align: center :alt: PyScripter - Running an install command from the PyScripter console Once the software is installed, go to the WAPT Console, then find your development host. In the :guilabel:`Software inventory` tab find the software title and copy the value indicated in the uninstallkey column. .. figure:: wapt-resources/wapt_console_uninstallkey-select_screen-item.png :scale: 75% :align: center :alt: Retrieving an uninstallkey from the WAPT Console Retrieving an uninstallkey from the WAPT Console You **MUST** also check the value of the version with the value indicated in :code:`min_version` in your :file:`setup.py`. Modify your :file:`setup.py` with the new parameters: .. code-block:: python # -*- coding: utf-8 -*- from setuphelpers import * uninstallkey = [] def install(): print('installing tis-firefox-esr') install_exe_if_needed("Firefox Setup 45.5.0esr.exe",silentflags="-ms",key='Mozilla Firefox 45.5.0 ESR (x64 fr)',min_version="45.5.0") To test that your key works correctly, you **MUST** run an installation again in :program:`PyScripter`. .. image:: wapt-resources/windows_pyscripter_run-install_menu-item.png :align: center :alt: PyScripter - Running an install command from the PyScripter console WAPT should not attempt to install the software because it is already present, the following message should display: .. code-block:: python :emphasize-lines: 6 >>> *** Remote Interpreter Reinitialized *** Command Line : install "c:\waptdev\tis-firefox-esr_x64_PROD_fr-wapt\WAPT\.." Using config file: C:\Program Files (x86)\wapt\wapt-get.ini Installing WAPT files c:\waptdev\tis-firefox-esr_x64_PROD_fr-wapt Exe setup Firefox_Setup_78.7.1esr.exe already installed. Skipping Results: === install packages === c:\waptdev\tis-firefox-esr_x64_PROD_fr-wapt | tis-firefox-esr (78.7.1-102) Now you can now test the uninstallation: .. image:: wapt-resources/windows_pyscripter_run-remove_menu-item.png :align: center :alt: PyScripter - Running a remove command from the PyScripter console You can now build and upload your package, please refer to the :ref:`documentation for build and upload packages from the WAPT Console `. .. note:: If you leave the uninstallkey blank, uninstalling your package will not work. Special case of a non-silent uninstaller ---------------------------------------- In some particular cases, a package using :command:`install_exe_if_needed` fills in the *uninstall key*, but the *uninstall key* points to a non silent uninstaller. We have to circumvent that problem by using a function that will remove the *uninstall key* at the end of the installation. .. code-block:: python :emphasize-lines: 13 # -*- coding: utf-8 -*- from setuphelpers import * uninstallkey = [] def install(): install_exe_if_needed("setup.exe", silentflags="/s", key='{D9E87643-0005-447E-9111-78697A9C1595}', min_version="14.0") uninstallkey.remove('{D9E87643-0005-447E-9111-78697A9C1595') def uninstall(): run(r'"C:\Program Files\Kutl\uninstall.exe" /supersilent') .. hint:: The uninstall feature can also be used to run code in addition to uninstalling software, ex: delete folder, delete shortcut ... Video demonstration ------------------- .. youtube:: z_EN2CBCTcY :align: center :width: 50% .. _packaging_empty_packages: Packaging empty packages ======================== .. _build_upload_from_console: Building the package and sending it to the WAPT Server ====================================================== * Once the package is ready, build it and send it to the WAPT Server. .. image:: wapt-resources/wapt_console_build-upload_menu-item.png :align: center :alt: Menu option *build-upload* in the WAPT Console * Select the package in the :file:`c:\\waptdev` folder. .. figure:: wapt-resources/wapt_console-build-upload-select-folder_browser-window.png :scale: 75% :align: center :alt: Browser window for selecting the WAPT package to import into the private repository Browser window for selecting the WAPT package to import into the private repository * Confirm the selected package. .. figure:: wapt-resources/wapt_console_build-upload-confirm_dialog-box.png :align: center :alt: WAPT Console dialog box for confirming the importation of a WAPT package into the private repository WAPT Console dialog box for confirming the importation of a WAPT package into the private repository You have just uploaded your first wapt package. .. note:: :ref:`A command line method is available here `. .. warning:: Once your package has uploaded, refresh the package list using the :guilabel:`Refresh packages list` button or by pressing :kbd:`F5` on your keyboard. .. _returncodes: Working with non standard return codes -------------------------------------- Return codes are used to feed back information on whether a software has installed correctly. In Windows, the standard successful return code is [0]. If you know that your WAPT packages installs correctly, yet you still get a return code other than ``[0]``, then you can explicitly tell WAPT to ignore the error code by using the parameter :code:`accept_returncodes`. You can find out how to use the :code:`accept_returncodes` parameter by exploring this package code. .. code-block:: python :emphasize-lines: 30 # -*- coding: utf-8 -*- from setuphelpers import * import re uninstallkey = [] def is_kb_installed(hotfixid): installed_update = installed_windows_updates() if [kb for kb in installed_update if kb['HotFixID' ].upper() == hotfixid.upper()]: return True return False def waiting_for_reboot(): # Query WUAU from the registry if reg_key_exists(HKEY_LOCAL_MACHINE,r"SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate\Auto Update\RebootRequired") or \ reg_key_exists(HKEY_LOCAL_MACHINE,r"SOFTWARE\Microsoft\Windows\CurrentVersion\Component Based Servicing\RebootPending") or \ reg_key_exists(HKEY_LOCAL_MACHINE,r'SOFTWARE\Microsoft\Updates\UpdateExeVolatile'): return True return False def install(): kb_files = [ 'windows10.0-kb4522355-x64_af588d16a8fbb572b70c3b3bb34edee42d6a460b.msu', ] with EnsureWUAUServRunning(): for kb_file in kb_files: kb_guess = re.findall(r'^.*-(KB.*)-',kb_file) if not kb_guess or not is_kb_installed(kb_guess[0]): print('Installing {}'.format(kb_file)) run('wusa.exe "{}" /quiet /norestart'.format(kb_file),accept_returncodes=[0,3010,2359302,-2145124329],timeout=3600) else: print('{} already installed'.format(kb_file)) if waiting_for_reboot(): print('A reboot is needed!') .. hint:: The full list of Windows Installer Error Messages can be found by visiting `this page `_.