Integrating Subversion with Mantis Bug Tracker using SCMBug

From BrushpenWiki

Jump to: navigation, search

Contents

Overview

This tutorial covers the setup of WAMP (Windows, Apache, MySQL, PHP/Perl/Python and/or Primate), Subversion Server version control, and Mantis bug reporting, as well as the steps necessary to integrate these applications using an external Software Configuration Management "gluing" application, SCMBug.

WAMP

Subversion and Mantis both require an Apache web server with the appropriate modules installed. Additionally, Mantis requires PHP and MySQL. WAMP provides a convenient one-stop solution to provide all of these applications on Windows.

Installation

Get WAMP here, run the setup, and follow the default instructions. Note that the WAMP\www\ directory is the default location that web applications such as Mantis will be published to. Make sure this directory is in an appropriate location for backup purposes.

Subversion

Installation

The standalone Subversion binaries include the modules needed to run Subversion on an Apache server (TortoiseSVN does not). Get the Windows Subversion Server binaries here. TortoiseSVN provides easy to use shell integration, and can also be used to easily create a repository. Get TortoiseSVN here.

Apache Modules

There are a total of five Apache modules that need to be installed and enabled to access a Subversion repository through the web. The first two are for WebDAV (Distributed Authoring and Versioning) and are provided by the WAMP installation. They will need to be enabled in the httpd.conf file or through the WAMP system tray menu. The standalone WebDAV modules are:

  • mod_dav.so
  • mod_dav_fs.so

The next two are provided by the standalone Subversion binaries and are located in the application\bin folder (e.g. C:\Program Files\Subversion\bin. They can be referenced in the httpd.conf file from this location or copied to the wamp\Apache2\modules folder and referenced there. These files are:

  • mod_dav_svn.so
  • mod_authz_svn.so

In order to set up Windows authentication for Subversion, Apache needs a Security Support Provider Interface (SSPI) module The latest mod_mod_auth_sspi can be downloaded here. As of 8/31/2006, mod_auth_sspi-1.0.3-2.0.58.zip should be the appropriate version for WAMP5 running Apache 2.0.58. All that is needed is the mod_auth_sspi.so, which can be extracted to the wamp\Apache2\modules folder.

Configuration

httpd.conf

Access the Apache httpd.conf file by left-clicking the WAMP system tray icon and selecting Config files > httpd.conf. This file contains the Apache server configurations, and the Apache service will need to be restarted for any changes to take affect.

LoadModule Directives

The following LoadModule directives need to be added/uncommented to the Dynamic Share Object section of the httpd.conf file:

LoadModule dav_module modules/mod_dav.so
LoadModule dav_fs_module modules/mod_dav_fs.so
#Or wherever the svn modules are located
LoadModule dav_svn_module "D:/Program Files/Subversion/bin/mod_dav_svn.so"
LoadModule authz_svn_module "D:/Program Files/Subversion/bin/mod_authz_svn.so" 
LoadModule python_module modules/mod_python.so
LoadModule sspi_auth_module modules/mod_auth_sspi.so

Some of these, such as dav_module, dav_fs_module, and sspi_auth_module may already have entries that are just commented out/disabled. Removing the "#" and the front of the entry will enable them.

<Location> Property for HTTP Repository Access

Once a Subversion repository has been created (see the TortoiseSVN Manual for details), a virtual location needs to be added to the Apache web server to provide http access to the repository. Append the following to the bottom of the httpd.conf file, where repository_dir is the name of the folder that will show up on the

<Location /repository_dir>
  DAV svn
  #Location of the folder containing the repsotiory root folder
  SVNParentPath "D:/Dev/TestRepos"
  # authentication
  Require valid-user 
  AuthType SSPI
  AuthName "Subversion Authentication"
  #The file containing authorized users, stored in the wamp\Apache2\conf directory
  AuthzSVNAccessFile conf/svnaccessfile.conf
  SSPIAuth On
  SSPIAuthoritative On
  #The Windows domain to authenticate against
  SSPIDomain DOMAINNAME
  SSPIOfferBasic On
</Location>

SVNParentPath must be the location that contains the folder that the repository files are located in. In the example above, D:/Dev/TestRepos contains another folder such as /ToolsRepository, which in turn contains the /conf, /dav, /db, /hooks, /locks, format, and README.txt that are created when a Subversion repository is created. SVNParentPath cannot be the actual location of the repository files. See below for information on creating the svnaccessfile.conf.

SVNAccessFile / Windows Authentication

The SVNAccessFile defines the groups that have access to the repository, as well as the read/write permissions for that group. Following is an example svnaccessfile.conf placed in the wamp\Apache2\conf directory:

[groups]
admin = DOMAINNAME\hhiggins, DOMAINNAME\jdoe, DOMAINNAME\dgerman
# Default access rule for ALL repositories
# Everyone can read, admins can write
[/]
* = 
@admin = rw

See External Sources for more advanced access file configurations.

Mantis

Prerequisites

MySQL

Before running the Mantis installation, it is necessary to configure the MySQL server included with the WAMP installation. To do this, browse to the included virtual directory (alias) http://localhost/phpmyadmin/ (may not display if trailing / is not provided). Click on the Privileges link and configure the accounts that will be used by Mantis to access the MySQL database.

PHP Mail support

As of WAMP5_1.6.4, PHP mail server support comes disabled out of the box. Because of this, Mantis's default behavior of emailing new users their registration password will not function. Mantis may encounter the following error:

PROBLEMS SENDING MAIL TO: test123@hotmail.com
Mailer Error: Could not instantiate mail function. 

There are two fixes for this:

  • Disable the password reset process of Mantis (new users will be created with a blank password). Copy the following from mantis/config_defaults_inc.php to mantis/config_inc.php, near line 150. Set g_send_reset_password to "OFF":
# if ON users will be sent their password when reset.
# if OFF the password will be set to blank. If set to ON, mail settings must be
# correctly configured.
$g_send_reset_password   = OFF; 
  • Enable the mail support extensions in PHP
1.  Open the php.ini from the tray icon Config Files > php.ini
2.  Configure the [mail function] variables for an available SMTP server:
[mail function]
; For Win32 only.
SMTP = somesmtpserver.domain.com
smtp_port = 25
; For Win32 only.
;sendmail_from = firstname.lastname@brushpen.net

Note that the sendmail_from variable must be a valid account on the specified SMTP server, or the email will bounce. Contact your network administrator for SMTP server information and to open port 25 for the computer that is hosting Mantis. Additionally the mantis\config_defaults_inc.php lists several variables that should be customized:

# --- email variables -------------
$g_administrator_email	= 'administrator@example.com';
$g_webmaster_email = 'webmaster@example.com';
# the 'From: ' field in emails
$g_from_email = 'noreply@brushpen.net';
# the return address for bounced mail
$g_return_path_email = 'admin@example.com';
3.  Save and close the file and refresh the PHP extensions from the task menu.

Installation

Get the latest version of Mantis here. Extract the files into the wamp\Apache\www directory (or the publish directory that was specified during the WAMP installation). Browse to the http://localhost/ and follow the link to the mantis folder listed under Your Projects, or browse to the address of the mantis folder directly (e.g., http://localhost/mantis/). Specify the username and password to be used to access/write to the database for bug tracking operations, the username and password for installation (root or other administrator), and the name of the database. Make note of the database name, as this will be needed later for some version control-bug tracking applications.

Configuration

Once Mantis is installed, return to the mantis directory in a web browser, and log in as the administrator to change the settings. The default password is NOT blank, as suggested by the form.

Default Mantis Administrator:
 Username: Administrator
 Password: root

Use the Manage link to change the Administrator password, specify mantis users, define projects and required fields and other configurations.

SCMBug

Prerequisites

Cygwin

In order to install SCMBug, Subversion must provide a diff.exe as part of its verification process. This is unfortunately, as Subversion's diff process is handled by 'svn diff'. A suitable substitute is provided in Unix and is available in Windows using Cygwin tools. Use the installer to download diffutils package, and the copy the diff.exe from C:\cygwin\bin to the bin folder of the Subversion installation.

Perl Packages

SCMBug requires an installation of Perl. ActivePerl was recommended by the SCMBug documentation. In a command console type:

C:\>ppm
PPM>install Mail-Sendmail
PPM>install XML-Simple (installed with default WAMP configurations)
PPM>install DBI
PPM>install DBD-mysql

Provide the yes to install on any packages that are not installed, do nothing for ones that are.

Installation

Get the latest version from http://www.mkgnu.net/. Extract the files to C:\Program Files\SCMBug. Note that the project contains absolute path references to this location and will not work elsewhere.

Daemon.conf

The first step after the files are in place is to configure the daemon to run on the local system. The daemon will listen on port 3872 by default. This port may need to be opened for incoming traffic by IT. For Mantis in the deployment scenario described with default values, the following entries need to be provided:

# Configuration information of the bug tracking
# system
bugtracker => {
# The possible options are:
# - 'Bugzilla'
# - 'Mantis'
# - 'RequestTracker'
type => 'Mantis',
# This MUST be in the format x.y.z, where x,y,z
# are integers
version => '1.0.5',
database_location => '127.0.0.1',
database_port => '3306',

Of course these settings may change depending on Mantis version and whether the Mantis database is on the same system. The next piece is the database information, which was configured during the Mantis installation.

# Valid values are the ones accepted by the Perl DBI.
#
# For Bugzilla and RequestTracker, this value is ignored
#
# For Mantis, *some* valid values (there are others) are:
# - 'mysql'
# - 'Pg'
database_vendor => 'mysql',
database_name => 'bugtracker',
database_username => 'MySQLUserName',
database_password => 'MySQLPassword',

The next value of interest is the bug_url_prefix, which needs to have a format like this:

# Prefix used to display a bug
bug_url_prefix => 'http://127.0.0.1/mantis/view.php?id='

Pay close attention that all commas and curly braces are preserved in the original .conf file.

The final step for this configuration file is to create the domain user to mantis user mappings. These will be of the format:

# This is a one-to-one mapping of SCM usernames to bugtracking
# usernames. Mappings in this list override mappings from
# mapping_regexes.
mapping_values => {
    enabled => 1,
    values => {
	'DOMAINNAME\\hhiggins' => 'hhiggins',
	'DOMAINNAME\\jdoe' => 'jdoe',
	'DOMAINNAME\\dgerman' => 'dgerman'
	}

Note that the last user is not followed by a comma. Note that the value for enabled is set to 1 - the template has it set to 0. Do not forget to have a comma after each earlier mapping definition.

Now, to avoid obvious problems - like deamon is not running - start the scmbug deamon by executing:

C:\Program Files\Scmbug\etc\init.d>scmbug-server.bat

To check if the deamon correctly started - just go to:

C:\Program Files\Scmbug\var\log\scmbug\activity.log

and look for errors - if nor error - all is ok.

Gluing to Subversion

The next step is to run an included install perl script that will embed the appropriate hook scripts into the target repository. From a console, brose to C:\Program Files\SCMBug\bin. The scmbug_install_glue.pl script will require several parameters:

C:\PROGRA~1\Scmbug\bin>scmbug_install_glue.pl --scm=Subversion --product=AVPToolkit 
--repository=file///D:/Dev/TestRepos/Test1 --bug=1 --binary-paths="D:/Program Files/S
ubversion/bin,D:/Wamp/www/mantis" --daemon=127.0.0.1
  • --scm is the version control system to be used.
  • --product is a string that will be checked against the project category of the mantis case. By default, if the product specified here does not match the project specified for the specific Mantis case, the daemon will not allow the check-in to take place.
  • --repository This is the location of the existing Subversion repository that will have the gluing hooks installed. SCMBug will not overwrite any existing hook scripts if they exist.
  • --bug The Mantis case SCMBug will attempt to log the gluing process in.
  • binary-paths is the location of the version control system, and optionally, the path of the Mantis files (necessity of the Mantis path needs to be verified)
  • --daemon is the IP address of the system that the daemon will be running on. If the repository and daemon are on the same system, this can be localhost or 127.0.0.1.

Once the installation is successful, the pre-commit and post-commit scripts will be launched any time a check-in occurs. These are located in the hooks folder of the repository.

Glue.conf

After the install scmbug_install_glue.pl script is run, it is necessary to tweak a few final configurations in the code that runs when a check-in occurs to enforce the policies for that specific project (see the PowerPoint presentation at mkgnu.net for details on supported policies. This glue.conf file will be located in the repository's hooks\etc\scmbug\ directory.

The glue_configuration should have 'Subversion' set as the scm name:

#
# Describes the SCM system integrated with bug-tracking
#
scm => {
name => 'Subversion',
label_directories => [
 'Tags',
 'Branches'
 ]

The log_template defines the regular expressions that validate the check-in log. The following listing provides a regular expression that validates a log of the format: MantisID #4: Log message here.

log_template => {
   # The bugid_regex is a regular expression that must set
   # the unnamed variable $1 to the bug number, or list of
   # bug numbers. It is checked for a match as: m/$regex/s
   bugid_regex => 'Mantis\s*ID\s*(.*?):',
          # The bugid_split_regex is a regular expression describing
          # how a list of bug ids will be split in individual bug
          # numbers. It is split as: /$regex/
          bugid_split_regex => ',\s?#|\s?#|,|\s+',
   # The log_regex is a regular expression that must set the
   # unnamed variable $1 to the log comment. It is checked
   # for a match as: m/$regex/s
   log_regex => 'Mantis\s*ID.*?:\s*(.*)'
   }, 

This regular expression allows the script to break apart the Mantis case number and report it to the daemon, which will in turn log the change list in mantis under this case number.

By default, the valid_product_name will require the project specified in Mantis is the same as the product specified when the scmbug_install_glue.pl script was run. The following overrides that and supports a project structure based on the existing repository structure.

The next three options are worth mention, as they define policies such as whether a non-owner can check in under a specific Mantis ID, whether a check-in can occur if the case is not open, and the message log length:

# The SCM user issuing an activity must be the user to which
# the bug is assigned
valid_bug_owner => {
    enabled => 1
    },
# The bug against which an activity is issued must be in an
# open state
open_bug_state => {
    enabled => 1
    },
# Minimum number of characters log message.
minimum_log_message_size => {
    enabled => 1,
    size => 25
    },
# Valid product name. There are 2 options:
#
# - type is 'manual'. Each bug id supplied during commit
#   messages must be filed against the product name
#   specified in value.
#
# - type is 'auto'. The product name will be autodetected. It
#   will be computed as the directory name that follows a
#   directory prefix from the list of directory prefixes
#   specified in values. The list of directory prefixes should
#   be comma(,)-separated.
valid_product_name => {
    enabled => 0,
    type => 'auto',
    value => 'Trunk,Utilities'
    },

Potential Setup Issues

  • If the SCMBug Daemon cannot be found, the user will be unable to check in
  • If the Mantis case is not assigned to the user, they cannot check in

SCMBug Errors and Troubleshooting

  • Daemon crashes after one check-in - terminates abnormally
This appears to be an issue with the daemon not returning to a stable state after the first check-in. This may be a bug with the 0.8.24 release of the application, which seems to have had minimal testing done with Subversion and Mantis.
  • Subversion log does not meet regular expression requirements
Currently, SCMBug dumps an error stating that the log did not match the log format regular expressions. It may be appropriate to customize the regular expressions in the glue.conf file as well as the related SCMBug source to provide a more display-friendly error. This could include a usage example.

External Links

Downloads

WAMP
Apache 2.0 modules, including mod_auth_sspi
Windows Subversion Binaries
TortoiseSVN
Mantis

External Sources

SVNServe Apache Configurations
SCMBug Manual
Getting SVN working with WebDAV on Windows…
Windows Authentication with Subversion on Windows
Mantis Forum: Doesn't let me to create user and doesn't send email

Personal tools