Stacksmith

How to add new types of Parts to Stacksmith

2019-01-05T10:21:57+00:00

It is probably easiest to just take an existing part type and duplicate and rename all its classes. I recommend using “button” or “timer”.

Every part needs to have a unique one-word “type” string that can be used for its part type in the XML and for distinguishing it from other part types when listing and accessing parts via other object descriptors than just “part”. So pick one now. For example, buttons have “button” and movie players have “moviePlayer”.

When should I add a new part vs. a new style of an existing part

Stacksmith and HyperCard before it have a long history of subsuming what would be distinct control types as styles of one part. The main criterion applied here should be what it feels like to the user. Implementation under the hood should not dictate breaking out a new kind of button into something else than a button style. But conversely, just because some objects behave like re-painted versions of each other should not mean they should be the same part type. A good example of this are sliders and scroll bars, which are used for completely different things, so should be different part types, not just styles of the same scale part.

A good indicator of a part that should be broken out is often if it suddenly needs to add properties to a part type that have nothing to do with the other part styles. Like the minValue/maxValue properties on SuperCard’s scrollbar button style, that none of the others would need. A good example of integrating a completely different control as a style are simple list fields. Users think of these as lists, and write lists into text fields, then think they might want to be able to select a line, so turn on autoSelect and implicitly get a platform-native table view. However, it’s also an edge case, as multi-column table views exceed this metaphor and might call for a different implementation.

Registering instructions for a part type

All object descriptor syntax is defined by host function tables in WILDHostFunctions.cpp. First, add new enum entries to WILDHostFunctions.h. You’ll usually need 4 instructions: WILD_CARD_typename_INSTRUCTION, WILD_BACKGROUND_typename_INSTRUCTION, WILD_NUMBER_OF_CARD_typenameS_INSTRUCTION and WILD_NUMBER_OF_BACKGROUND_typenameS_INSTRUCTION. These let a script enumerate and access your objects on the card and background layers.

Next define the corresponding functions (and their prototypes) in WILDHostFunctions.cpp. They are just a small shim that calls through to an internal function that does all the actual work and takes the type string as their parameter. E.g. WILD_CARD_FIELD_INSTRUCTION is implemented as:

void	WILDCardFieldInstruction( LEOContext* inContext )
{
	WILDCardPartInstructionInternal( inContext, "field" );
}

There are corresponding WILDBackgroundPartInstructionInternal, WILDNumberOfCardPartsInstructionInternal and , WILDNumberOfBackgroundPartsInstructionInternal functions that you can use to implement the other three instructions.

Now that you have the instruction functions, register them with the instruction table lower in the file by adding LEOINSTR() entries for them in between the LEOINSTR_START() and LEOINSTR_LAST() macros. Note that they must be in the same order as in the header’s enum. If your instruction was added at the end of the list, ensure that you change the previous last item into a regular LEOINSTR() and make your last item the new LEOINSTR_LAST().

Registering syntax for a part type

Once you have defined the instruction functions and the symbolic constants you’ll refer to them with, you need to define the actual Hammer syntax that scripters will use to refer to your part. A typical object descriptor looks like card button "foo" or background field id 1. Some may also use several words like background movie player "double feature". In addition, a part may be specified in relation to its containing object, like card timer 7 of card "timekeepers".

#Registering new identifiers

First, you must make sure that any new identifiers you need are defined in ForgeTypes.h in the IDENTIFIERS macro (most identifiers like card, of and the like are already defined, so usually all you need to add is the identifiers for the singular and plural form of your object.

To just define a single identifier, say “frog”, use

X1(EFrogIdentifier,"frog")

Where the first parameter is the constant that will be used by you in host code (the “token identifier type”). The second parameter is the actual string the user will use in a Hammer script. This string must be all lower-case. Some older xTalks permitted short forms for identifiers as synonyms that can be typed more quickly. It is generally not recommended to add short forms (users read scripts more often than they write, so they should rather use autocompletion to write the longer identifiers than write a short form that they have to mentally expand when reading every time), but should you need to, you can define a synonym using the X2 macro:

X2(EFrgIdentifier,EFrogIdentifier,"frg")

This declares a new token EFrgIdentifier that is equal to the string “frg”. When the tokenizer generates the token list for parsing, all EFrgIdentifier tokens will automatically be changed into EFrogIdentifier tokens.

Note that this is not a general synonym specification mechanism. Tokens declared as synonymous like this will not be converted back into their short form by the parser. There is no way you can match EFrgIdentifier alone in some spots once it is a synonym for another token. It is recommended to define other synonyms simply by providing two kinds of syntax. This mechanism is simply there to support classic HyperTalk synonyms like card/cd etc.

#Registering the actual syntax

The actual syntax for an object descriptor is defined in WILDHostFunctions.cpp in the gStacksmithHostFunctions array. The entries in this table are checked in order. It is recommended you group entries that start with the same identifier together and put the ones that have the most identifiers before the first parameter above the ones that have fewer or an expression right after the first identifier. The Forge parser can currently only backtrack over identifiers, which means that if it parses the object descriptor card button "foo" encounters the syntax entry for card "foo" first it can not backtrack once it has matched button as a variable name, and will thus present a syntax error before it ever gets to the case for card button "foo".

There are six entries for each part type’s syntax: “card typename x”, “background typename x”, “number of card typenames”, “number of background typenames” and the shorter “typename x” equivalent to “card typename x”, and “number of typenames” equivalent to “number of card typenames”. Just duplicate the entries and the array will resize to accomodate them.

Creating the actual class that implements a part

There are generally two classes. The cross-platform class, e.g. CButtonPart, and a platform-specific subclass that actually implements a part’s physical appearance in platform-native API. On the Mac, that would be CButtonPartMac, which is a subclass of both CButtonPart and CMacPartBase, which implements some common behaviours and API specific to the Mac.

CButtonPart uses tinyxml2 API to load and save itself from disk, and defines properties to hold this information and the code to allow scripts to access its attributes.

CButtonPartMac defines the CreateViewIn() and DestroyView() methods that the subclass should override to (re)create a Macintosh NSView or dispose of it and remove it from display. In addition, it implements methods like GetPropertyEditorClass() and GetDisplayIcon() that return Mac stuff specific to this object that the UI needs to properly display the various editors. It also implements some convenience methods that convert certain typical Stacksmith internal data types into Mac types and back, and that provide canned implementations of typical Mac implementations for cross-platform methods, like CMacPartBase::OpenScriptEditorAndShowOffset(), which you should call from your subclass’s CPart::OpenScriptEditorAndShowOffset() override to make sure scripts can ask your part to show its script editor and it will show a Mac window containing the script editor.

Once you have the class, you have to make the platform-specific CStack subclass aware that it should use this subclass when it encounters a part of your type. On the Mac, you add a line like this to CStackMac’s RegisterPartCreators() static method:

CPart::RegisterPartCreator( new CPartCreator<CButtonPartMac>( "button" ) );

Where “button” here is the unique type string you picked for your part type.

Adding UI for creating your part

Currently, there is no standard mechanism for creating new parts. You’ll have to duplicate the CStackMac’s -newButton: method rename and change it to create your part type, and create a menu item for it in the MainMenu.xib and hook it up to the -newButton: action on First Responder, and make sure your part can cope with being created without having been given any XML to load. Hopefully we can eventually use XML templates so your part can get a sensible default size and default attributes from there, and we can generate the “New X” menu items automatically.

How to add new commands to Stacksmith

2019-01-04T10:21:57+00:00

To define a new command in Stacksmith, you first define a few instructions that implement these commands and register them using LEOAddInstructionsToInstructionArray, then you define the syntax in a table and pass that to LEOAddHostCommandsAndOffsetInstructions. That’s it.

How to register new instructions

First, define a header file for your command to hold some symbolic constants for the instructions you need:

enum
{
	MY_FROBNITZ_INSTR = 0,
	MY_FROBOZZ_INSTR,
	MY_FROBAR_INSTR,
	
	NUM_FROBNITZ_INSTRUCTIONS
};

and forward-declare an array of instruction function pointers:

LEOINSTR_DECL(Frobnitz,NUM_FROBNITZ_INSTRUCTIONS)	// Declares gFrobnitzInstructions global.

Then create a matching implementation file and declare a function that implements the behaviour for each of your instructions. An instruction function looks like:

void	MyFrobnitzInstruction( LEOContext* inContext )
{
	// Look ar inContext->currentInstruction to see param1 and param2
	// Last on-stack parameter will be in inContext->stackEndPtr -1, second-to-last at -2 etc.
	
	inContext->currentInstruction++;	// Advance to the next instruction in the script.
}

Then create the actual definition of your instruction array, one each corresponding to the constants in your enum:

LEOINSTR_START(Frobnitz,NUM_FROBNITZ_INSTRUCTIONS)
LEOINSTR(MyFrobnitzInstruction)
LEOINSTR(MyFrobozzInstruction)
LEOINSTR_LAST(MyFrobarInstruction)

To make Leonie aware of these instructions, you need to add the instructions to Leonie’s internal table of instructions. Leonie already knows some built-in instructions, and you may have already added another batch of instructions for a different command, so you will need to know at which position these instructions have been inserted. We define a global variable to hold this offset:

size_t						kFirstFrobnitzInstruction;

and also add an extern declaration to our header. Now you can go to the host application’s main() function (or in the case of Stacksmith, to the -initializeParser method of the WILDAppDelegate and actually register your instructions:

LEOAddInstructionsToInstructionArray( gFrobnitzInstructions, NUM_FROBNITZ_INSTRUCTIONS, &kFirstFrobnitzInstruction );

If you were to manually generate your bytecode, you could now add kFirstFrobnitzInstruction to e.g. MY_FROBNITZ_INSTR to get the actual LEOInstructionID to write into the bytecode so Leonie will execute the correct function.

How to add new command syntax

Imagine you wanted to add a command whose syntax was:

frobnitz <text> [frobozz <otherText>]

where the second parameter and its ‘frobozz’ label were optional. Stacksmith does not yet know the identifiers ‘frobnitz’ and ‘frobozz’, so the first thing you do is go to ForgeTypes.h. At the start of this file, there is a #define IDENTIFIERS that defines a mapping between the constants representing an identifier (e.g. EFunctionIdentifier) and the actual string it corresponds to in the source file (in lowercase, because Forge converts every character to lowercase before it compares them, thus giving the illusion of a case-insensitive programming language). Go to the last line of that define. Let’s say it was

X1(EPlayIdentifier,"play")

Add two more lines so it reads:

X1(EPlayIdentifier,"play") \
X1(EFrobnitzIdentifier,"frobnitz") \
X1(EFrobozzIdentifier,"frobozz")

If you wanted “frob” to be a short form for “frobozz”, you could also add an entry:

X2(EFrobIdentifier,EFrobozzIdentifier,"frob")

Which will map all occurrences of the word “frob” to the EFrobozzIdentifier, too. Now that Forge knows our new identifiers, we can define a syntax table for our command, in a global variable:

struct THostCommandEntry	gFrobnitzCommands[] =
{
	{
		EFrobnitzIdentifier, MY_FROBNITZ_INSTR, 0, 0, '\0',
		{
			{ EHostParamExpression, ELastIdentifier_Sentinel, EHostParameterRequired, INVALID_INSTR2, 0, 0, '\0', '\0' },
			{ EHostParamLabeledExpression, EFrobozzIdentifier, EHostParameterOptional, MY_FROBOZZ_INSTR, 0, 0, '\0', '\0' },
			{ EHostParam_Sentinel, ELastIdentifier_Sentinel, EHostParameterOptional, INVALID_INSTR2, 0, 0, '\0', '\0' }
		}
	},
	{
		ELastIdentifier_Sentinel, INVALID_INSTR2, 0, 0, '\0',
		{
			{ EHostParam_Sentinel, ELastIdentifier_Sentinel, EHostParameterOptional, INVALID_INSTR2, 0, 0, '\0', '\0' }
		}
	}
};

The first line defines the command’s initial identifier as EFrobnitzIdentifier, so it starts with “frobnitz” as we want it. It also tells the parser to use the WILD_FROBNITZ_INSTR instruction when this command is matched during parsing. The next two zeroes are the param1 and param2 fields of the instruction, in which you can pass additional information for your instruction function. The following array holds one entry for each parameter, its end marked by a parameter whose type is set to EHostParam_Sentinel. Each parameter entry is defined as:

type, identifier, optional, instructionID, param1, param2, modeRequired, modeToSet

The first parameter in our example is of type ‘immediate value’, meaning the first parameter can be any expression (which fulfills our “text” criterion well enough, it could be a string concatenation expression, after all). This expression will be pushed on the stack before our instruction function is called. Since we only have an expression and no identifier labeling it, we pass ELastIdentifier_Sentinel as the identifier to mean ‘we don’t care’. The first parameter is required, so we say that here. We don’t want to change the instruction we compile to, so we pass INVALID_INSTR2 here, and 0 for param1 and param2.

The second parameter is a labeled expression. That is, an identifier as a label, followed by another expression. The identifier for the label is of course Frobozz, and it is optional. Now, I’ve arbitrarily decided that although the Frobozz variant of our Frobnitz command looks fairly similar, it will actually be implemented by a completely different instruction, MY_FROBOZZ_INSTR. If I specified INVALID_INSTR2 here as well, both variants of the command would be handled by the MY_FROBNITZ_INSTR instruction. If the second parameter is present the second item on the stack will be the second expression. If the second parameter is left out, an empty string will be pushed instead. However, since I specified a different instruction to be used, if the second parameter is missing, the parser will not push a second parameter on the stack. Only when it is present will MY_FROBOZZ_INSTR be called with two parameters on the stack.

Since we only define one command in this example, the second command simply starts with ELastIdentifier_Sentinel, indicating this is the end of the array.

Now all that’s left is registeringyour new command’s syntax with Forge. To do that, call

LEOAddHostCommandsAndOffsetInstructions( gFrobnitzCommands, kFirstFrobnitzInstruction );

sometime at startup, ideally right after you register the Frobnitz instructions. Pass in the kFirstFrobnitzInstruction global, so Forge can offset all the instruction IDs you specify correctly. Of course this also means that you can only use instructions in this syntax entry that you registered together.

If your syntax is more complex, you can take advantage of the modeRequired and modeToSet fields of parameters. A command starts out in mode ‘\0’. It will match any parameter whose modeRequired is ‘\0’ as well. If you specify any other character in the modeToSet of a parameter that matches, it will from then on only look for subsequent parameters that have the same character in modeRequired. In addition, you can specify a required terminal state at the top with the command’s name. Usually, ‘X’ is used here. If parsing the command ends, but the mode is not this character, parsing will be considered a failure. This is useful to define parameters as optional only when another parameter is specified instead. If neither is specified, the second one will not set the state to ‘X’, and parsing will fail as desired, even though both are optional.

Writing a simple instruction function

Just as an example, let’s implement a typical instruction function like our example above would use it:

void	MyFrobozzInstruction( LEOContext* inContext )
{
	LEOValuePtr	stackParam2 = inContext->stackEndPtr -1;	// Was pushed last, so at end of stack.
	LEOValuePtr	stackParam1 = inContext->stackEndPtr -2;
	
	char	stackParam1StrBuf[1024] = { 0 };
	const char*	stackParam1Str = LEOGetValueAsString( stackParam1, stackParam1StrBuf, sizeof(stackParam1StrBuf), inContext );

	char	stackParam2StrBuf[1024] = { 0 };
	const char*	stackParam1Str = LEOGetValueAsString( stackParam2, stackParam2StrBuf, sizeof(stackParam2StrBuf), inContext );
	
	DoAnActuallFrobozz( stackParam1Str, stackParam2Str );
	
	LEOCleanUpStackToPtr( inContext, inContext->stackEndPtr -2 );
	
	inContext->currentInstruction++;	// Advance to the next instruction in the script.
}

First, this calculates the position on the stack of our two parameters. Then it retrieves these values as strings. If they are actual string values, this will return the actual internal string pointers in stackParam1Str and stackParam2Str. If they are other values, it converts them to strings in the stackParam1StrBuf resp. stackParam2StrBuf buffers we provide and returns those in stackParam1Str and stackParam2Str instead.

Once we’ve extracted the 2 strings, we call the function that does the actual work, DoAnActualFrobozz() and hand the two strings to it (if Frobozzing wasn’t such a complex task, we’d probably just do it right there in the instruction function).

Lastly, we remove our 2 parameters from the stack by unwinding the stack by 2 slots, and advance to the next instruction. This is what pretty much all command instructions would do. Only if you are implementing a branching instruction, would you add or subtract different values from the currentInstruction pointer of the context.

How to add functions to Stacksmith

2019-01-03T10:21:57+00:00

There are three ways to implement functions in Stacksmith:

Built-in functions - These look and are called like any function handler implemented in Hammer, but actually compile to an instruction you define. However, they are limited to taking no parameters, and you can currently only add one by directly editing the built-in function table in CParser.cpp.
Host functions - These follow fairly freeform syntax, and are defined like host commands, using a table you pass to LEOAddHostFunctionsAndOffsetInstructions, and compile to an instruction as well. The return value of the function must be pushed on the stack by your instruction after all parameters have been removed.
Provide a callNonexistentHandlerProc to the LEOContext and at that point handle any function for which no handler exists in the current message path. The number of parameters and the parameters themselves will have been pushed on the stack, and can be accessed using the utility function LEOGetParameterAtIndexFromEndOfStack. Once you are done, use the utility function LEOCleanUpHandlerParametersFromEndOfStack to remove your parameters from the stack.

See How to add commands to Stacksmith on how to create your own instructions, and how the general process of specifying a syntax table work in Stacksmith.

How to Add Properties to Stacksmith

2019-01-02T10:21:57+00:00

There are 3 steps involved in adding a new persistent property to an existing object in Stacksmith (usually implemented as an object that is a subclass of CConcreteObject or one of its subclasses):

Find the CConcreteObject-conforming C++ class that implements the object to which you want to add a property.
Define a C++ instance variable on the object to hold the data at runtime. Write a setter for it that calls IncrementChangeCount() to make sure any change to that property gets saved. Find the LoadPropertiesFromElement() method and add code to load the property from the file, and add code to write the property back out again to the SavePropertiesToElement() method.
Add code that makes the property accessible to scripts to GetPropertyNamed() and SetValueForPropertyNamed(), keying off the (case-insensitive!) property name for scripts. You will also want to add code to log the property’s value to DumpProperties() as an aid when debugging.

Road Map

2019-01-01T10:21:57+00:00

A project like Stacksmith is a huge undertaking. To make it achievable, it will be split up into smaller milestones that should result in regular releases that will already be useful to a subset of its target audience.

Road Map

Game Engine

The first milestone is intended to be a simple 2D game engine. Enough to create a point-and-click adventure, a train set, or maybe a tower defense game. HyperCard already is very graphical, so lends itself to such games, and most games live in a single window and have little platform-native UI, which means most of that can be postponed for a later milestone. Main features needed for this are:

The editor needs to allow creating and assigning icons, and resizing and positioning elements has to work well. We need working undo. Otherwise creating a game with Stacksmith would be a chore.
We need animation. This includes just flipping through icons with code, visual effects for navigating cards and hiding/showing objects, but also bezier paths so we can move an object along its points.
We need sound output. Ideally a bug-free “play” command that plays melodies, but just playing a sound sample would be enough for this milestone. We’d also need support for importing sounds in the editor.
We need support for saving state. Either by implementing file access, or by saving changes to a stack (which would be un-editable in a shipped game) as a diff somewhere, or by making sure a second stack can be created and accessed using scripts.
This could include support for draw graphic objects (see below)

Personal Database

The next milestone would be to make sure Stacksmith is suitable for use as a database.

Most of this would probably just be performance optimizations to stack navigation and loading. Also over the web.
We’d need a fast search, probably using an index.
We’d need a working “sort” command
We’d need to make sure the “mark” and “go next marked card” etc. commands work reliably and quickly.
We’d need support for the user changing media used by parts (e.g. to make an image database that either imports or references images or sounds)
This could include support for showing lists based on contents in cards.
This could include support for printing.
This could include support for generating reports that can be printed.

Standalones

We already have support in Stacksmith that it opens any stack named “Home.xstk” in its Resources folder. Now we’d need UI for building a standalone from a stack by creating a copy of Stacksmith itself and copying the file into it. Bonus points if we’d then properly codesign it.
We’d need support for creating your own menu bar full of menus. This includes an editor as well as syntax and saving the objects to the stack and to make a menu bar current in some way.
We’d need support for customizing the about screen at the least, multiple windows (i.e. multiple stacks per project) at best. But the latter would require some hard decisions about syntax.

Drawing Application

We need vector shapes like ovals, rectangles, rounded rectangles, lines, bezier paths (= polygons, too), images, groups.
Vector shapes would need to be optimized so you can have many of them and they’ll still load and draw quickly, e.g. by only creating one actual UI object for a sequence of draw graphics that don’t have scripts anyway.
Boolean operations? Clipping?
Add UKPaintView back in, both for the card picture and for editing images in-line?
Export of graphics (both the entire card’s appearance, or individual images to write them back out after editing)
Filters?
Groups should work with all objects, not just with graphics.

Custom UI elements

It should be possible to create your own custom UI elements and replicate them without having to update each manually whenever you fix or extend them. This should include buttons, fields (in-line objects, but I doubt we can allow customizing the text layout engine), and list/table cells. Ideally also custom menu item views.

Bootstrapping

Eventually, it would be great to have a version of Stacksmith that could be used to build Stacksmith. At the least, any bit of UI that Stacksmith has built in should be possible to implement using stacks and scripts, but then be customizable. Selecting buttons and showing selection handles should be scriptable and it should be possible to extend the “fake editor” UI built this way by looking at properties and handling messages.

Advanced IDE

All during this process, we will try to add modern IDE features.

More graphical programming (a la Prograph/Marten, Scratch or Automator, with an eye towards Button Tasks and SuperCard’s ClickScript)
Support programming by discovery by adding small iPhoto-style pop-overs that help you experimentally find out what messages to handle to get a certain result, show you what messages are happening in the app etc.
Focus on forgiveness. Add code that copes with less logical but more human expressions like “add 15 to the selectedLine of me” instead of “add 15 to line (the selectedLine of me) of me” etc.
Add a “stack canvas” that shows the entire project and all objects in it, but can also be locked down and filtered to be a quick game map.
Add more “programming by example” stuff, if we can come up with stuff that we can detect and offer to do for the user.
Add more feedback to the code editor, including parsing the code, showing all variables, and showing results of the scripts in a nice way (without actually executing the scripts and thereby modifying the user’s stack). Like Bret Victor’s stuff.

HyperCard Clone

At some point, we might put an effort into making Stacksmith more compatible with HyperCard stacks.

Special modes for imported HyperCard stacks that switch rects to being comma-delimited lists, styles in fields to be more like the old ones etc.
Special visual appearance for parts so they look more like they did in HyperCard
We could even add modes that anti-alias old icons so they look better, or that don’t anti-alias any of our part graphics (including text!) so they look more like the old stuff.
Add the AddColor command (as a built-in command, we already parse the data in our file format), so old color stacks look right.
We could implement compatibility code for certain popular XCMDs.
If we want to go overboard, we could add a minimal OS 9 “emulation” that would run XCMDs.