MAPaint Technical Report:
Power Assist

Power Assist

Power assist is the term coined for using image processing techniques to help the painting process. This ranges form simple erosion and dilation of the painted region to automatic tracking of structures through the volume. The list of IP support tools is currently:

Geometry object:

this is not yet implemented but will allow the user to place simple geometry objects (circle, square, etc) on the image. This is intended for applications such as marking dots, nuclei etc where the presence and distribution is the information of interest and not the detailed shape.

Threshold:

select a point and the domain to add (or subtract) is defined by the set of pixels with grey value within a certain range of values around the selected pixel grey value and connected to the selected pixel by other members of the set. The range can be adjusted by dragging the mouse right and left and the resulting new domain is interactively displayed. Releasing the mouse button accepts the changes. By pressing the left mouse button with the <control> key pressed the mode is switched to allow the user to define a constraint domain within which the thresholded domain is restricted. This allows holes in an otherwise complete boundary to be blocked.

Fill:

the displayed section can be considered to be segmented into number of regions according to the existing painted domains. In this sense ``no-paint'' can be considered a region. The fill option will add (or subtract) the singly connected region containing the selected point. The tool controls with fill allow the user to select between 4- and 8-connectedness to define the connected regions. 8-connected tends to allow too much ``leaking'' and therefore 4-connected is the default.

Erode, Dilate:

apply erosion or dilation to the current domain with a structuring element which can have different sizes and distance metrics. The size options are of a radius between 1 and 10 and the distance options are 8-connected, 4-connected, octagonal and Euclidean. Pressing left mouse button over a region will erode it, <control> left mouse-button will dilate it. In this way the user can execute combinations of morphological operations on the defined domains within the 2D section. These are useful for defining thin boundary regions and for smoothing.

Affine transform:

allows affine transform (currently only translate and rotate) of previously defined regions. The tool controls allow selection of which type of transform. In the current version the choice is between translate and rotate. To move a domain component click the left mouse-button within the required domain and drag. In translate mode the domain can now be moved into place. In rotate mode (indicated by the cursor shape) the point at which the mouse-button was pressed will act as the centre of rotation and the domain will rotate by the angle between an imaginary lever from the centre of rotation to the cursor and the x-axis. For small rotations drag out a longer lever to the right hand side of the centre of rotation.

Image Tracking:

modify the domain propogated from the previous operation (usually the previous section) by using the image information associated with that domain to match to the existing image. The method matches small parts of the image near to the domain and establishes the best match at a series of points along the domain boundary. This is closely related to the ``snake'' techniques developed for flexible model matching but with a number of differences. The first is that there is no global assumption about the image match cost, the matching at each point on the boundary depends on the image used to define the initial boundary. This breaks the requirement that the snake must have the same appearence at all parts. The second is that there is no constraint on the absolute curvature of the boundary, only a constraint on the change of curvature with respect to the original. This allows the preservation of specific sharp features in the boundary which may be real. There is a control panel which allows the modification of a set of control parameters for the searched area and the snake cost function. The cost function is optimised using Dynamic Programming and is an approximation to the snake cost (making this optimisation globally correct is straightforward and will be implemented soon).

Edge Tracking:

this is essentially identical to image-tracking but now the cost is derived from the local edge strength which is currently defined as the modulus of the gradient. The gradient is calculated using a derivative of gaussian filter with an adjustable width. The gaussian suppresses high frequency noise. This edge strength is not ideal because it does not include the directional component and it is possible to result in a new domain which switches between two closely parallel edges of opposite polarity.

Tablet:

in many cases it is easier and more convenient to draw on printed pictures of the sections which can be done using a graphics tablet. This is useable simply as a means to control the cursor, the use of a pen to draw is more natural and easier to control than a mouse, but the main use is to draw from the printed sections. To link tablet coordinates with image coordinates the user defines two fixed points on the tablet which are transformed to two given points within image space. In our case the fixed points are generated as crosses on the printed picture and are typically at opposite ends of a diagonal. The user first marks each point then draws around the required region. The tool controls allow the user to define the image coordinates of the reference points. These points could of course match fiducial markers within the sectioned material if they exist. The tablet controls are for a Wacom cordless tablet.¹

Domain Review:

allows the user to loop throught the segmented parts of a given 3D domain and add that region to a selected domain by pressing a button. The selection also include return to grey, delete, and discard which will leave any regions under the reviewed region unchanged. The domain to be reviewed can be read in directly or may be generated within the program. For example it is possible to select ``Check Overlap'' on the read domain dialog which will then check the domain read for overlaps with any existing domain. If an overlap is found then the overlap region is registered with the domain-review controls which when activated allow the user to check each overlap region in turn and select the required destination domain. Alternatively the review domain can be read directly from disc.

The domain-review control dialog provides a complete list of the non-NULL planes within the review-domain object and selection of a plane will change the view to that plane. Pressing ``review'' initiates the review from the selected plane and continues until all planes are complete or ``Stop'' is pressed. In review mode the region to be reviewed is flashed on the screen and the user simply selects which domain that region should be added to. Very small regions (less than 10 pixels) are highlighted by adding a large square around the region which also flashes. It is not required that all the parts of a review-domain are allocated but the review-domain dialog must be dismissed before other a painting tools can be used. Any remaining regions are saved for later review unless overwritten by an operation that can result in a review-domain (read domain).

Domain Surgery

implemented but no description.

The tracking tools have only recently been implemented and need tuning to establish the best default parameters for different applications.