/*-------------------------------------------------------------------------------
  cawi_popro.ado: Stata post-processor for NEPS CAWI raw data, wrapping -casi_popro-

    Copyright (C) 2023-  LIfBi | Survey Technology (surveytechnologie@lifbi.de)
                            Daniel Bela

    This program is free software: you can redistribute it and/or modify
    it under the terms of the GNU Affero General Public License as published
    by the Free Software Foundation, either version 3 of the License, or
    (at your option) any later version.

    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    GNU Affero General Public License for more details.

    You should have received a copy of the GNU Affero General Public License
    along with this program.  If not, see <https://www.gnu.org/licenses/>.

-------------------------------------------------------------------------------*/
*! cawi_popro.ado: Stata post-processor for NEPS CAWI raw data, wrapping -casi_popro-
*! Daniel Bela (surveytechnologie@lifbi.de), Leibniz Institute for Educational Trajectories (LIfBi), Germany
*! version 2.2.2 13 May 2024 - new directory structure also for LS modules; a little more chatty error messages
*! version 2.2.1 12 April 2024 - minor bugfixes for CAWI mode
*! version 2.2.0 08 April 2024 - reworked directory structure for output
*! version 2.1.1 27 March 2024 - add '*.sql' files from TBT root directory to TBT export data
*! version 2.1.0 20 March 2024 - add overview dataset with testgroup/id-mappings
*! version 2.0.10 04 March 2024 - explicit subdirectory for ZEUS logging archives
*! version 2.0.9 26 February 2024 - allow focus person type C
*! version 2.0.8 08 February 2024 - auto-rename merged id variable based on target focus person identifier
*! version 2.0.7 09 January 2024 - make temporary extract paths optional arguments
*! version 2.0.6 09 January 2024 - report 'lastpage' for dropped, empty observations, if possible; make dropping empty observations an option
*! version 2.0.5 18 December 2023 - tolerate leading/trailing spaces around tokens when handling Matomo tokens
*! version 2.0.4 30 November 2023 - bugfix: no redundant type conversion before appending datasets
*! version 2.0.3 14 November 2023 - added robustness against empty Matomo or exit survey datasets
*! version 2.0.2 06 October 2023 - allow focus person type I
*! version 2.0.1 03 August 2023 - reworked detection of episodical variables (and datasets)
*! version 2.0.0 31 July 2023 - implemented handling if instrument-wise Matomo exports, including auto-fixing Matomo tokens
*! version 1.9.0 29 July 2023 - correctly process exit survey data (paradata from force-canceling or timeout events)
*! version 1.8.0 14 July 2023 - updated routine for changed file structure in A105, introduced logic to auto-separate address data from regular data
*! version 1.7.1 12 Mai 2023 - bugfix handling of auto-dropped observations
*! version 1.7.0 19 April 2023 - don't try to export / process datasets without observations; include the ZIPped raw data in output directory
*! version 1.6.4 19 April 2023 - bugfix in handling string list of dropped ids
*! version 1.6.3 02 March 2023 - added failsafe mechanism to make sure observations dropped due to lack of valid data in one data source can safely be dropped from another, even if dropping in this other datasource would be skipped due to no observations left
*! version 1.6.2 28 February 2023 - add '*.db' files from TBT root directory to TBT export data
*! version 1.6.1 28 February 2023 - do not drop observations without valid information in case the result would be an empty dataset file
*! version 1.6 24 February 2023 - added explicit routine to export LimeSurvey timing data
*! version 1.5 31 January 2023 - added CAWI wrapper -cawi_popro-
*! version 1.4 20 December 2022 - added sub-option -nofakeids- to -id_list-, and default mechanism to create fake ids; remove requirement for target id to be unique in -id_list-; be more robust when bulk erasing files (wait 1s and retry if a single rm call fails)
*! version 1.3 19 December 2022 - added routine to raw-copy TBT content data recursively
*! version 1.2 16 December 2022 - bugfix update: tolerate $ signs in file or directory names, tolerate non-unique tokens in source data
*! version 1.1 07 December 2022 - reduced version requirement to Stata 16.1
*! version 1.0 23 November 2022 - added usage message (full help file will follow)
*! version 0.6 22 November 2022 - added file name convention mechanics
*! version 0.5 22 November 2022 - added inserting export host name and time stamp, and option to omit it
*! version 0.4 22 November 2022 - added option to insert version number into file names
*! version 0.3 21 November 2022 - added detection and extraction of episodical data
*! version 0.2 16 November 2022 - added automatic id change integration
*! version 0.1 02 November 2022 - initial development release
program define cawi_popro, nclass
    version 16.1

    **# save syntax
    syntax , [ * ]

    **# invoke cawi_popro with option cawimode
    if ( missing(`"`options'"')) {
        casi_popro
    }
    else {
        casi_popro , `macval(options)' cawimode
    }
    exit 0
end
**# EOF