Command line Archives

Monitoring the Web Publishing Engine in FileMaker Server 11

May 31, 2012 by Anders Monsen 2 Comments

FileMaker Server 11 includes several components to publish databases to client machines. The Database Server is perhaps the most obvious component, as this hosts the databases that are shared with clients.

Any type of web publishing requires the Web Publishing Engine (WPE) to be active and running. Custom web publishing such as PHP communicates to the Database Server via the WPE. But what happens when a web page is no longer is responsive? Which process failed? Is it a page error, a Database Server problem, or an issue with the WPE? As you can imagine, manually checking the state of the server becomes tedious.

The ability to build an automatic monitoring method becomes critical for anyone managing web pages that communicate with FileMaker Server, to rapidly identify problems and minimize downtime.

The Admin Console is a GUI-based tool to manage databases, backup and script schedules, clients, statistics, and the current state of the database and WPE processes. However, as it is a GUI-based system, it needs eyes-on management. In order to automate the process we need to turn to some system-based tools.

Turning to the Command Line

One oft-used method is to create a PHP page that pings the Database Server. If the ping – a PHP request – returns a valid result, then all is well. If the ping returns an error, the administrator intervenes. However, when setting this up, I ran into issues with PHP sendmail failures, and I wanted a process that minimized manual intervention.

To fully automate the process I used one set of tools to check the state of the WPE, and another to automatically switch on the WPE if it wasn’t running. Although FileMaker Server comes with a command line interface – fmsadmin – there is no documented process to manage the WPE in FileMaker Server 11.

When FileMaker, Inc. released Server 12 on April 4, 2012, new options were added to start and stop processes; now you can stop and start the WPE process via fmsadmin. However, if you’re running FileMaker Server 11, you need to take advantage of some undocumented processes. (Here the usual caveat applies: test to make sure this works under your circumstances).

In the Mac OS X environment you can use Unix commands to check running processes. One such command “nc” lets you scan for listening daemons to check if a process is running on a specific port, without sending data. By scanning a WPE port you then can see if this is active for a given IP address.

nc -zw 3 127.0.0.1 16008

This command will return nothing if the process is inactive. If the process is running, the command returns a text string with a success message.

Once we’re able to run such a command in the Terminal, the next step is to set this up in a shell script. If we first check for the WPE and find that it isn’t running, we next verify that the Database Server is running. Once we know this process is active, we then run a sequence of fmsadmin commands. These I gleaned from a website (thanks to Andrew Duncan from Databuzz) that provides a preference pane to control FileMaker Server 11. The key command is:

fmsadmin start|stop adminserver|wpc|cwp

When I tried this out in the Terminal, I found that it only would work for me when I ran them on separate lines, and in a certain sequence.

fmsadmin start wpc fmsadmin start cwp

After all this is combined into a shell script, you can run this as a cron job on Mac OS X. I simply saved the code below into a wpe.sh file, made this executable and set up the cron to check the process. The #lines are comments that tell me what happens inside each branch, and the if statements check to see whether there is a value returned from the query or not.

#!/bin/bash #wpe running up_wpe=`nc -zw 3 127.0.0.1 16008`
if [ -z "$up_wpe" ]; then #echo "wpe is not running"
  up_fms=`nc -zw 3 127.0.0.1 5003`
if [ -n "$up_fms" ]; then #echo "DBS running"
  fmsadmin start wpc fmsadmin start cwp fi fi

This probably is an un-orthodox method to check the WPE process and start it up, but it appears to work. With FileMaker Server 12 you still need to step outside FileMaker Server’s built-in tools to check the process, but you’re able to run the documented “fmsadmin start wpe” command. Possibly there are other ways to monitor the WPE process, and refinements to this method. We’d love to hear any that have worked for you.

ImageMagick and FileMaker

February 2, 2012 by Anders Monsen 4 Comments

A few months ago I wrote a piece on image manipulation in FileMaker container fields on Mac OS X. That type of manipulation was relatively simple – rotating an image – but required AppleScript and ImageEvents. What if you required a more complex adjustment of images, such as converting from TIFF to JPEG, or compression, blurring, re-sizing?

Image Events actually does some of these types of manipulation, but not always with expected results. Many people turn to Photoshop for these effects. You can also write AppleScripts to interact with Photoshop and FileMaker, but may end up dealing with syntax changes or application version changes. Photoshop also must be open and active, and is expensive.

Another option we looked into was ImageMagick, a powerful and free open source command line tool that also will work with multiple programming language like PHP, Perl, Ruby and others.

Installing ImageMagick

Installing ImageMagick on the Mac requires time and patience. Apparently the people who support this (free, by the way) software only maintain the latest OS-compatible version as an executable installer – meaning Mac OS Lion. The machines that would run this were still on Snow Leopard, so this required a more manual method.

The Mac OS X binary release requires MacPorts, which in turn requires XCode. These first must be installed and configured correctly. MacPorts, once installed, also required a self-update (sudo port -v selfupdate), which in our instance required tweaking rsync settings due to firewall issues. Alas, this step is poorly documented, but in our instance we changed this document: /opt/local/etc/macports/source.conf

By commenting out the last line, which uses rsync, and adding a direct url, we were able to get MacPorts to update properly.

# rsync://rsync.macports.org/release/tarballs/ports.tar [default]

http://www.macports.org/files/ports.tar.gz [default]

Then in the Terminal, we typed the following. Then we sat back and enjoyed many minutes of peace while ImageMagick installed, as this took a while.

sudo port -d sync

Finally, installing ImageMagick itself was a one line command in the Terminal:

sudo port install ImageMagick

Once installed, you can test ImageMagick by running “convert-version” in the Terminal. If properly installed, you will get something like the following:

"Version: ImageMagick 6.7.2-0 2011-09-20 Q16 http://www.imagemagick.org Copyright: Copyright (C) 1999-2011 ImageMagick Studio LLC Features: OpenMP OpenCL"

Integration with FileMaker

Since it now worked in the Terminal, it was time to turn our attention to integration with FileMaker. For this to work we needed to execute a shell statement from FileMaker. As this solution already used Troi File, which has a function to execute shell, we just needed our command. Troi File’s command TrFile_ExecuteShell( ) is used to send commands to ImageMagick. Prior to each shell action we built the command in a variable, which let us debug this in the Data Viewer.

To complicate matters, certain ImageMagick actions returned values, while others returned nothing. For example, the version command returned the string inside the execution step, so setting a variable to TrFile_ExecuteShell populated this variable with the result. This was a perfect way to test whether or not to continue, since computers without ImageMagick installed would not be able to perform anything, and we could exit gracefully if the version command did not contain any ImageMagick information.

There are several commands that we can send to ImageMagick. Each command accepts different arguments. For example, “identify” provides information about the image, while “convert” performs the actual conversions. The shell command string requires the full path to the command, which usually is inside the “/opt/local/bin/” folder.

Our primary requirement was to convert an image from one type to another, filtering and resizing the image to certain specifications. The beauty of ImageMagick and shell scripting is that you can point to a source image across the network and send the converted file to another location across the network.

The convert command uses certain parameters such as source file, resize and resize value, and where to write the file. An example of this with some fill-in-the-blank naming appears below.

"/opt/local/bin/convert path_to_source/filename.TIF -set option:filter:blur 0.8 -resize %RESIZE_VALUE% -strip -quality 90 -write \"path_to_destination/filename.jpg\" +delete"

Alas, this command returns nothing from within FileMaker. There is no way to know whether or not the conversion was successful, aside from a second command using Troi File to see if the new file exists.

Conclusion

While there are significant cost and overhead advantages to using ImageMagick over, for example, Photoshop, there is a corresponding cost in terms of time and effort. The command line doesn’t always invite users to have a great experience.

While I have not tested these features, Monkeybread Software’s plugin for FileMaker comes with image editing using GraphicsMagick, a fork from ImageMagick a few years back. Other plugins provide execute shell commands, so you’re not just limited to Troi File.

Trapping for errors remains important, as sometimes you get feedback, and sometimes nothing, from within FileMaker. Usually you can copy the command from the Data Viewer into the Terminal, and then you will get some feedback about any errors. That said, this is a powerful method to integrate image manipulation into FileMaker.

Monitoring the FileMaker Server xDBC Process

November 9, 2011 by Anders Monsen 17 Comments

Sending data to FileMaker from external sources poses interesting options and challenges.

One of our development projects required enabling ODB/JDBC and sending data via JDBC (Java Database Connectivity). The sending application used FileMaker’s client driver, which allowed it to send SQL insert statements to the FileMaker database via FileMaker Server. This required Server Advanced, and enabling ODBC/JDBC in the server settings:

This process worked seamlessly, as long as FileMaker’s “fmxdbc_listener” process remained active. But, in our non-perfect world, sometimes this process failed.

Even though the configuration setting remained checked in the Admin Console, the process simply stopped, and our database no longer would accept incoming connections from the Java app. Unless someone happened to watch the process, hours or days could pass before anyone noticed this problem. We needed a method to monitor the fmxdbc_listener process. (Since the servers run on Mac OS X, this solution is not cross-platform.)

Activity Monitor showed all FileMaker Server processes, when filtered by “fm”:

While the Activity Monitor provided a good user interface to see all system processes, we needed to automate and script the scenario.

The command line script was something that could be automated. Using “ps” we could view different processes. Targeting one specific process required adding “grep”, such with this command:

ps aux | grep '[f]mxdbc_listener'

This generated an output when the process ran:

fmserver   624   0.0  2.3   213008  97120   ??  S     9:49AM   0:00.27 ./fmxdbc_listener

If the process went offline, the Terminal showed an empty process. This could then be written as a shell script, or in our case, a PHP command line script with the option to send an email.

#!/usr/bin/php -q

<?php
  error_reporting(0);

  $fmx = exec("ps aux | grep '[f]mxdbc_listener'");

  if(empty($fmx)) {
    $to      = 'admin_group@server_name.com';
    $subject = 'XDBC Listener unavailable';
    $message = 'Reset process';
    $headers = 'From: admin@xdbc_server.com' . "\r\n" .
      'Reply-To: admin@xdbc_server.com' . "\r\n" .
      'X-Mailer: PHP/' . phpversion();

    $mail = mail($to, $subject, $message, $headers);
  }
?>

When we moved the code to the server it failed to generate an email. Since the server resided inside a firewall, we switched to an internal FileMaker process instead. Using the PHP API, we triggered a FileMaker script in a monitoring/logging database, which sent the email via SMTP, a process we already used on other databases on that server and knew would work.

#!/usr/bin/php -q

<?php
  error_reporting(0);

  require_once('FileMaker.php');
  define('FM_HOST', 'IP_Address');
  define('FM_FILE', 'xdbcLog.fp7');
  define('FM_USER', 'web');
  define('FM_PASS', 'web');
	
  $fm = new FileMaker( FM_FILE, FM_HOST);
  $fm->SetProperty( 'username', FM_USER );
  $fm->SetProperty( 'password', FM_PASS );
  $fmLayout = 'Log';
  $scriptName = 'Send_eMail';
	
  $fmx = exec("ps aux | grep '[f]mxdbc_listener'");
	
  if(empty($fmx)) {
    $script = $fm->newPerformScriptCommand( $fmLayout, $scriptName);
    $result = $script->execute();
  }
?>

To automate this process we enabled it as a cron job, checking the server every few minutes. To reactivate the process, you could simply uncheck the ODBC configuration, save your action, check it again, and save once more. So, although human intervention is required to restart the process, setting up this system script send an immediate alert when the process is detected as offline. Whether this is foolproof method for process-monitoring remains to be seen. Why the process fails is another question. If other FileMaker developers have encountered similar issues, I would be interested in hearing how you have dealt with this issue, both on Mac and Windows servers.

Turning to the Command Line

Installing ImageMagick

Integration with FileMaker

Conclusion

Let’s get started on your project.