1. Sending Form Data from FileMaker to a Web Service

Web services come in many different formats, be it JSON, XML, or simple HTML form data. The latter, though comprehensive to post from FileMaker, is rather easy to post from an HTML page with fields for manual data entry.

  1. Sending Form Data from FileMaker to a Web Service
    1. Speed Comparison
  2. Modifications after rewritten to ACF-Plugin version 1.7.0

Example:

Let's assume we have an HTML form like this:

<!DOCTYPE html>
<html>
<head>
    <title>Sample Form with File Upload</title>
</head>
<body>
    <form action="target_url_here" method="POST" enctype="multipart/form-data">
        <label for="name">Name:</label>
        <input type="text" id="name" name="name" required><br><br>

        <label for="address">Address:</label>
        <input type="text" id="address" name="address" required><br><br>

        <label for="city">City:</label>
        <input type="text" id="city" name="city" required><br><br>

        <label for="zip">Zip:</label>
        <input type="text" id="zip" name="zip" required><br><br>

        <label for="email">Email:</label>
        <input type="email" id="email" name="email" required><br><br>

        <label for="file">Upload a File:</label>
        <input type="file" id="file" name="file"><br><br>

        <input type="submit" value="Submit">
    </form>
</body>
</html>

The POST request sent to the web server looks like this:

Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW

----WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="name"

John Doe
----WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="address"

123 Main Street
----WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="city"

Sample City
----WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="zip"

12345
----WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="email"

johndoe@example.com
----WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="file"; filename="example.txt"
Content-Type: text/plain

[Contents of the uploaded file go here]
----WebKitFormBoundary7MA4YWxkTrZu0gW--

If the file(s) are binary, they usually need to be in base64-encoded format.

Sending data from FileMaker to a web service in this format can be challenging, especially when you have to attach Base64-encoded files to the post.

In this practical example, we need to send an invoice to a service to produce EDI documents in an Electronic Trade Format (EHF) for a customer, and we are using a web service to create the XML. The service accepts up to five attached PDF files with the invoice, including the invoice itself. To easily send this data, we have created a custom function to format the POST data as shown above and send it. We initially developed this solution some time ago, and now, I have recreated it with the ACF plugin to improve error handling and readability.

When it comes to error handling, ACF provides automatic exception handling. In the event of an exception occurring within the code, such as a FileMaker calculation returning a question mark (?), our previous custom function failed to detect and handle this appropriately. Consequently, it continued execution, leading to inaccurate results that were not easily discernible to the user.

In the new functions, I deliberately introduced an error to test the error handling mechanism. To my satisfaction, the system promptly responded with the following message:

Error message

Now, we can handle the situation and get correct result produced.

The error shown, was the return from the call to the function, so we check for the text ERROR in it....

 # Create and send the Invoice
Set Variable [$EHF; Value: ACFU_EHFService ($$ApiKey;......)]
 # Did it go well?
If [Position ( Left ( $EHF ; 100 ) ; "ERROR" ; 1 ; 1 ) > 0]
   Show Custom Dialog ("Error during EHF generation"; $EHF)
   Exit Script [Text Result: $EHF]
End If
...
...

Additionally, I have created a YouTube video explaining the functionality, including the Base64 encoding of binary files.

The ACF code is provided below, allowing you to copy and paste it into your own ACF libraries and customize it according to the web service requirements.

The example includes the use of two other plugins too. The TROI_Url, and Baseelements. 

package EHFsupportFunctions "Support functions for the EHF package 2";

function GetNameFromPath ( string path ) 
    FunctionID 3001;
    if ( isWindows ) then
        path = substitute ( path, "\\", "/");
    end if
    return regex_replace ("^(.*\/)?(.+\.[a-zA-Z0-9]{1,10})$", path, "$2"); 
end

/*
ACF functions and FileMaker calculation engine uses different variables. ACF variables is local to the plugin. 
When doing an assignment like '$$LastPost = postdata', the plugin in fact do a FileMaker calculation 
like: 

Let([$$LastPost = "<text content in postdata>"];"")

That calculation assign the value to the FileMaker variable. 
However, if the length of the text is more than 32K, this will fail due to limitations in string literals.   
We use a Getter function to pull this off instead as we asume the length of postdata will be considerably longer. 

Global declaration - Technique to handle large text assignment from function. 
*/
string LastEHFPost; 
// Getter function...
function GetLastEHFPost ()
    FunctionID 3501; // makes ACFU_GetLastEHFPost when called from FileMaker directly without ACF_Run.
    return LastEHFPost;
end


function EHFAttachement (string path, string Name, string Boundary )

    if ( path == "" ) then
        return "";
    end if
    
    string fn = GetNameFromPath(path);
    string v  = Boundary;
    v = v + format ('Content-Disposition: form-data; name="%s"; filename="%s"\r\n', Name, fn);
    v = v + "Content-Type: text/plain\r\n\r\n";
    
    // Use system commands to convert attachement to Base64. 
    string base64path;
    base64path = path+".base64"; 
    // Different commands for Mac and Windows
    if ( isMac ) then
        // Adjust absolute paths to match standard unix paths. 
        if ( left ( path, 8) != "/Volumes" && left ( path, 6) != "/Users" && left (path,1) == "/") then
            path = "/Volumes"+path; // Paths lack /Volumes as start... - like "/Macintosh HD...."
            base64path = "/Volumes"+base64path;
        end if  
        $$syscmd = format ('openssl base64 -in "%s" -out "%s"', path, base64path);
    else
        $$syscmd = format('cmd.exe /c certutil -f -encode "%s" "%s"', path, base64path);
    end if

    // Verify that the file exists. 
    if ( ! file_exists ( path ) ) then
        throw format ("ERROR: The Attached file '%s' at the path '%s' does not exists", fn, path ); 
    end if

    // Use Baseellements plugin to perform the system command
    $$SysRes = @BE_ExecuteSystemCommand ( $$syscmd ; 10000 )@;
    
    // Retrieve the base64 text
    int x = open (base64path, "r" );
    string base64 = read (x); 
    close (x); 
    if ( isWindows ) then
    // CertUtil makes a Certificate header and trailer that need to be removed. 
        base64 = between (base64, "-----BEGIN CERTIFICATE-----\r\n", "-----END CERTIFICATE-----"); 
    end if 
    // cleanup
    string res = delete_file ( base64path); 
    // return the part...
    return v + base64;
end

function FormField (string vb, string name, string content)
    return format ('%sContent-Disposition: form-data; name="%s"\r\n\r\n', vb, name) + content + "\r\n";
end


function EHFService ( string APIkey, string Bank, string FirmaNavn, string Adresse1, string Adresse2,
    string Postnr, string Sted, string OrgNr, string InvoiceBlock, 
    string AttPath1, string AttDesc1, 
    string AttPath2, string AttDesc2, 
    string AttPath3, string AttDesc3,
    string AttPath4, string AttDesc4,
    string AttPath5, string AttDesc5 )
    
    FunctionID 3502; // makes ACFU_EHFService when called from FileMaker directly without ACF_Run. 
    
    // Remove non-numeric characters from the Org-number
    OrgNr = regex_replace ( "[^0-9]",OrgNr, ""); 
    
    // format the post data to look like its sent from a HTML form. 
    string vb1 = "----WebKitFormBoundaryvaBIACFqQXnhryXmJxO";
    string vb2 = "--" + vb1 + "\r\n";
    string vb3 = "--" + vb1 + "--\r\n";
    string vUrl = "https://gw.ccx.no/ws/EHF20/";
    
    string h1 = "APIkey||Bank||FirmaNavn||Adresse1||Adresse2||Postnr||Sted||OrgNr" + Char(10); 
    string h2 = APIkey  + "||" + Bank + "||" + FirmaNavn + "||" + Adresse1 + "||" + Adresse2 + "||" + 
        Postnr + "||" + Sted + "||" + OrgNr ;
    
        // Build the Post...
    string postData = FormField ( vb2, "CompanyDetails", h1+h2) +
        FormField ( vb2, "InvoiceDetails", InvoiceBlock) +
        FormField ( vb2, "File1Desc", AttDesc1) +
        FormField ( vb2, "File2Desc", AttDesc2) +
        FormField ( vb2, "File3Desc", AttDesc3) +
        FormField ( vb2, "File4Desc", AttDesc4) +
        FormField ( vb2, "File5Desc", AttDesc5) +
        EHFAttachement ( AttPath1, "File1", vb2) +
        EHFAttachement ( AttPath2, "File2", vb2) +
        EHFAttachement ( AttPath3, "File3", vb2) +
        EHFAttachement ( AttPath4, "File4", vb2) +
        EHFAttachement ( AttPath5, "File5", vb2) + vb3; 
    
    
    // $$LastPost = postData; 
    // Use a different technique to assign the postdata to filemaker variable, as mention above. 
    
    LastEHFPost = postData; 
    string cc = @@Let([
        v = ""; 
        $$LastPost = ACFU_GetLastEHFPost
    ];
    v)@@;

    // Use Troi_URL plugin to send the data to the web-service. 
    string res = eval (format ('TURL_SetCustomHeader("-unused " ; "Content-Type: multipart/form-data; boundary=%s")', vb1) );
    string post = format('TURL_Post( "-TimeoutTicks=5000 -NotEncoded -Encoding=UTF8" ;"%s";$$LastPost)', vUrl); 
    string resData = eval ( post); 
    res = @TURL_SetCustomHeader("-unused "; "")@;
    return resData; 
end

1.1. Speed Comparison ↑

With both the new and old functions available simultaneously, conducting a speed comparison was a straightforward process.

The plugin provides two functions for measuring execution time: ACF_StartDurationTimer and ACF_GetDuration_uSec. To perform the test, I modified the script calling the functions as follows:

 # Measure the ACF Method
Set Variable [$t1; ACF_StartDurationTimer]
Set Variable [$EHF; Value: ACFU_EHFService ($$ApiKey;......)]
Set Variable [$t1; ACF_GetDuration_uSec]
 # then $t1 contains the number of uSeconds between start and get. 

 # To compare use the old standard custom functions:
Set Variable [$t2; ACF_StartDurationTimer]
Set Variable [$EHF; Value: EHFService ($$ApiKey;......)]
Set Variable [$t2; ACF_GetDuration_uSec]
 # Then $t2 contains the number of uSec for this method. 

 # Now we can present both.
Show Custom Dialog ["Time Consume:";"ACF Function: " & $t1 & ¶ & "Standard function: " & $t2]

It is anticipated that this process may take some time to execute, as it encompasses both the conversion of attachments to base64 format and the utilization of an external web service to generate the XML, in addition that we also are handling text in order of megabytes. The numbers are in microseconds.

I attempted the process multiple times, and the results consistently fell within the same time range. On each occasion, the ACF function executed twice as quickly. 0.25 seconds versus 0.5 seconds, approximately.

Speed compare

Considering that the response time for the web service and the base64 conversion remain consistent in both cases, when we account for these times during the comparison, it becomes evident that the ACF functions execute remarkably quickly. Although I haven't precisely measured these times, I estimate they would be at least 150-200 milliseconds. Let's assume 150 milliseconds, leaving us with 100 milliseconds for the ACF and 320 milliseconds for the standard custom function, making the ACF solution approximately three times faster. If it were 200 milliseconds, then the ACF time would be 50 milliseconds and the standard time 270 milliseconds, translating to roughly 5.4 times faster. This outcome is close to what we've seen in comparison to earlier studies.

2. Modifications after rewritten to ACF-Plugin version 1.7.0 ↑

With the new functions built into the plugin, we are not dependent on any other plugin anymore for this to work. It's both simpler implementation, and easier to follow.

Please note that in the ACF functions, the 'string' data type can accommodate both text and binary content. This is in contrast to the FileMaker 'TEXT' data type, which is not suitable for binary content due to the presence of special byte meanings within the 'TEXT' data type. In the 'string' data type used in the ACF plugin, all strings are bytes and they have a defined length. FileMaker employs 'Container Fields' for storing binary data, but there is no corresponding variable type for them. Therefore, within the ACF plugin, containers are not necessary, as we can work with strings.

Here is the source code for the 1.7.0 adjustments. (Of course, the old code also works...)

package EHFsupportFunctions "Support functions for the EHF package 3
This package requires plugin version 1.7.0 as it uses its built-in HTTP_POST and BASE64_ENCODE functions
instead of the base elements plugin and TROI_URL functions. 
";

function GetNameFromPath ( string path ) 
    FunctionID 3001;
    if ( isWindows ) then
        path = substitute ( path, "\\", "/");
    end if
    return regex_replace ("^(.*\/)?(.+\.[a-zA-Z0-9]{1,10})$", path, "$2"); 
end


function EHFAttachement (string path, string Name, string Boundary )

    if ( path == "" ) then
        return "";
    end if
    
    string fn = GetNameFromPath(path);
    string v  = Boundary;
    
    v = v + format ('Content-Disposition: form-data; name="%s"; filename="%s"\r\n', Name, fn);
    v = v + "Content-Type: text/plain\r\n\r\n";
    
    // Different commands for Mac and Windows
    if ( isMac ) then
        if ( left ( path, 8) != "/Volumes" && left ( path, 6) != "/Users" && left (path,1) == "/") then
            path = "/Volumes"+path; // Paths lack /Volumes as start... - like "/Macintosh HD...."
        end if
    end if
    
    // Verify that the file exists. 
    if ( ! file_exists ( path ) ) then
        throw format ("ERROR: The Attached file '%s' at the path '%s' does not exist", fn, path ); 
    end if
    
    // Retrieve the content 
    int x = open (path, "r" );
    string binary_content = read (x); 
    close (x); 
 
    // Convert to BASE64
    string base64 = BASE64_ENCODE (binary_content);

    // return the part...
    return v + base64+"\r\n";
end

function FormField (string vb, string name, string content)
    return format ('%sContent-Disposition: form-data; name="%s"\r\n\r\n', vb, name) + content + "\r\n";
end


function EHFService ( string APIkey, string Bank, string FirmaNavn, string Adresse1, string Adresse2,
    string Postnr, string Sted, string OrgNr, string InvoiceBlock, 
    string AttPath1, string AttDesc1, 
    string AttPath2, string AttDesc2, 
    string AttPath3, string AttDesc3,
    string AttPath4, string AttDesc4,
    string AttPath5, string AttDesc5 )
    
    FunctionID 3502; // makes ACFU_EHFService when called from FileMaker directly without ACF_Run. 
    
    // Remove non-numeric characters from the Org-number
    OrgNr = regex_replace ( "[^0-9]",OrgNr, ""); 
    
    // format the post data to look like its sent from a HTML form. 
    string vb1 = "----WebKitFormBoundaryvaBIACFqQXnhryXmJxO";
    string vb2 = "--" + vb1 + "\r\n";
    string vb3 = "--" + vb1 + "--\r\n";
    string vUrl = "https://mytestserver.com/ws/EHF20/";
    
    string h1 = "APIkey||Bank||FirmaNavn||Adresse1||Adresse2||Postnr||Sted||OrgNr" + Char(10); 
    string h2 = APIkey  + "||" + Bank + "||" + FirmaNavn + "||" + Adresse1 + "||" + Adresse2 + "||" + 
        Postnr + "||" + Sted + "||" + OrgNr ;
    
        // Build the Post...
    string postData = FormField ( vb2, "CompanyDetails", h1+h2) +
        FormField ( vb2, "InvoiceDetails", InvoiceBlock) +
        FormField ( vb2, "File1Desc", AttDesc1) +
        FormField ( vb2, "File2Desc", AttDesc2) +
        FormField ( vb2, "File3Desc", AttDesc3) +
        FormField ( vb2, "File4Desc", AttDesc4) +
        FormField ( vb2, "File5Desc", AttDesc5) +
        EHFAttachement ( AttPath1, "File1", vb2) +
        EHFAttachement ( AttPath2, "File2", vb2) +
        EHFAttachement ( AttPath3, "File3", vb2) +
        EHFAttachement ( AttPath4, "File4", vb2) +
        EHFAttachement ( AttPath5, "File5", vb2) + vb3; 
    
    // Send the data to the web service using our new HTTP_POST function. 
    
    // Make the headers for the request:
    string hdrs = format ( "Content-Type: multipart/form-data; boundary=%s",vb1 );
    
    // Then send the data
    string resData = HTTP_POST ( vUrl, postData, hdrs); 
    
    return resData; 
end