Serve clients' specific protocol requirements with Brazil, Part 2

How to support XML applications with the Brazil project

In Part 1 of this series, I introduced the concept of a developer-friendly experimental Web server technology called Brazil. The Brazil technology supports additional functionality via handlers, the simple Brazil Scripting Language (BSL), and a global dictionary of system and user properties. (For more information on BSL, properties, and handlers, refer to Part 1 of this series.)

The technology also supports languages that facilitate the rapid creation of XML content. In this month's installment of this series, I will explain how to represent different types of data in XML while still supporting those applications that lack XML ability. I will apply XML to three example applications' data: detailed realtime weather data, stock data, and data from other applications or devices. In addition, I will demonstrate how to use other languages, such as Python and Tcl scripting, with the Brazil Web server. This article will focus on XML generation, with future articles covering XML consumption.

Read the whole series on Brazil technology:

Benefits of XML

Numerous Webpages combine formatting and data. One of the primary benefits of XML is that it facilitates the separation of data from the formatting of that data. Good Web applications will support XML while providing support for those that don't.

Let's begin with a small example that demonstrates the coupling of data with formatting. Let's say we're interested in the collection and display of weather data. To make the temperature bold in an HTML table, we could write the following code:

 <td>Temperature <bold>76,5 </bold> </td>

To generate the page, eventually the data and code must be combined. For example, the above code segment could be realized in BSL by the following:

 <td>Temperature  <bold> <property temperature> </bold> </td>

Listing 1. Combine data and formatting in HTML via the typical approach

If you need to place the temperature on a page, the above approach is much easier than creating HTML each time by generating the content with a procedural programming language that requires recompilation for simple changes. You also don't have to search for a location for the temperature; simply put the <property ws.sample.Temperature> tag anywhere on the HTML page where you would like realtime weather data, and the Brazil technology will rewrite the page so that it will display the data when a browser displays it.

In prior articles, I configured the Brazil Web server to interface with a realtime weather station and provide Web data in the typical manner -- content plus data together -- as shown in Listing 1. For display to users, this approach is suitable, but more and more new Web applications consist of data from multiple Websites. Requiring developers that work on these new types of applications to parse HTML can lead to complications. To create a new service, developers would have to parse the data to obtain the weather information from site A to combine with site C's data. A process that separates the formatting from the content would solve these complications.

Click on GetSample.html at our weather station (http://www.brazilhandlers.com:9090/Applications/WeatherStation/ ) to use BSL to display weather station data. It will produce the following output:

weather.sampleRate               5000
weather.sample.Time              23:26:32 PM
weather.sample.Temperature       73.34
weather.sample.Time.UTC          970025192844
weather.sample.WindDirection      NOT CALIBRATED
weather.sample.WindSpeed          0.00

Listing 2. Use BSL to display weather station data

(Note: All of the examples in this article are available at http://www.brazilhandlers.com:9090/Applications/WeatherStation.)

The HTML page that utilizes BSL with the Brazil technology to generate the data in Listing 2 follows:

 <TABLE>
 <foreach name=a  match="(weather.sample.*)" >
         <tr>
                 <td> <property a.name.1> </td>
                 <td> <property a.value> </td>
         </tr>
 </foreach>
 </TABLE>

Listing 3. Generate the HTML from Listing 2

By substituting property values with real values in realtime, BSL improves the separation of formatting information and data. The BSL handler processes the simple lines above, and any properties matching weather.sample.* are assigned to the variable a and are available for page placement as shown by <td> <property a.name.1> </td>. The 1 represents the matched string as specified in most regex implementations.

The above technique is useful for generating your own pages, but quickly becomes tedious when you have to supply data to numerous users that all have different requirements. How do you accomplish this objective without staying up all night? You need to provide XML representations of the data and get out of the UI business -- in other words, you need to separate your data from content with XML. You must be pragmatic here, however. Your boss will not think too highly of your intelligence if your application supports XML yet doesn't meet the needs of those users lacking XML support. One of the strengths of the Brazil technology is its ability to live in both worlds, supporting both new and traditional technologies through task-specific handlers.

A formal XML primer would be beyond the scope of this article. Many books providing detailed examples are available to guide you (see Resources for help with XML). To generate XML tags, you need to examine your data elements; usually the nouns will be elements. In our example, temperature, wind speed, wind direction, precipitation, and barometric pressure are all nouns. For our weather station example, I am producing tags to simplify the XML. The sidebar at the Brazil Website features a real-world XML definition of weather data that defines a common language for the myriad of weather reports coming from ships, buoys, balloons, and manned and unmanned stations in different formats (see Resources for direct link). Before defining your own XML tags, search the Internet for preexisting definitions. Basically, you will search for a Document Type Definition (DTD).

Realtime weather data: A simple XML example

As explained in Part 1 of this series, our weather station can deliver data in a number of formats. Included among the supported formats is HTML, which features layout information, and a simple property format; applets consume data stored in the latter format to use in other applications. Data can also be exchanged via XML, which, as shown below, can be created by BSL.

Listing 4 defines a DTD for simple weather data. If you want to define your own XML tags, choose a DTD that defines the tags' values and specifies whether the elements are strings or tokens, and whether a specific element is mandatory or optional. Consult the XML specification for more details.

Several experts suggested the style featured in Listing 4, though some believe that XML DTDs should define tags that make it easier for computers to parse the tagged text. Specifically, developers often disagree whether to use elements and/or attributes. An element is the fundamental logical unit of an XML document. All content falls between elements. An attribute contains additional information about an element. Some designers prefer to place all the weather readings in one element with many attributes so that Temperature is an attribute of weather sample. More traditional programmers would use Temperature as an element. Some users find the single element with many attributes easier to read and easier for a computer to parse. The supporting Website in Resources contains examples of both, as BSL and Brazil support both versions. Here, I feature Temperature as an element.

<! ELEMENT MeteorologicalSample (Lat, Lon, Elev, Time, Temperature, 
WindSpeed, WindDiection)>
<!ELEMENT Lat (#PCDATA)>
<!ELEMENT Lon (#PCDATA)>
<!ELEMENT Elev (#PCDATA)>
<!ELEMENT Time (#PCDATA)>
<!ELEMENT Temperature (#PCDATA)>
<!ELEMENT WindSpeed (#PCDATA)>
<!ELEMENT WindDirection (#PCDATA)>

Listing 4. A DTD for simple weather data

Below is an example of how BSL can be used to generate XML:

<TAG>?xml version=<property weather.xml.version>?</TAG>
<TAG>DOCTYPE <property weather.xml.simple.title> SYSTEM <property 
weather.xml.simple.dtd></TAG>
<TAG><property weather.xml.simple.title></TAG>
        <Lat> <property weather.location.lat> </Lat>
        <Lon> <property weather.location.lon> </Lon>
        <Elev> <property weather.location.elev> </Elev>
        <Time> <property weather.sample.Time.UTC> </Time>
        <Temperature> <property weather.sample.Temperature> 
</Temperature>
        <WindSpeed> <property weather.sample.WindSpeed> 
</WindSpeed>
        <WindDirection> <property weather.sample.WindDirection> 
</WindDirection>
<TAG>/<property weather.xml.simple.title></TAG>

Listing 5. The .xml file, populated with real data via BSL

The following code segment is the result of using BSL to populate an XML template with real values. Note that each value -- including Temperature -- is an element, not an attribute.

<?xml version=1.0?>
<DOCTYPE SimpleMeteorologicalSample SYSTEM 
http://www.digiorgio.com:9090/SimpleMeteorologicalSample.dtd>
<SimpleMeteorologicalSample>
        <Lat> 41.222093 </Lat>
        <Lon> -73.317894 </Lon>
        <Elev> 100 </Elev>
        <Time> 970025759193 </Time>
        <Temperature> 72.93 </Temperature>
        <WindSpeed> 0.00 </WindSpeed>
        <WindDirection> NOT CALIBRATED </WindDirection>
</SimpleMeteorologicalSample>

Listing 6. The result of applying BSL to Listing 5

Realtime weather data: A real-world XML example

As I am no weather expert, I refer to a comprehensive weather data DTD that I found on the Web. More details on this DTD can be found in Resources.

Listings 7 and 8, which are located on the supporting Website http://www.brazilhandlers.com:9090/Applications/WeatherStation/JavaWorld, demonstrate how to use Brazil technology to comply with existing DTDs quickly.

Stock data

An overabundance of Websites sputter a constant stream of stock data. Take a peek underneath the browsers at the streams flowing into your machine. In most browsers, you can do this by selecting View Page Source. What you find is mind boggling: a massive amount of free-form scripting that combines user interface code and data. It works, but the data is not easily reusable. All Web content features this general problem; it is difficult to separate the content from the formatting in a deterministic way. A deterministic technique always provides you with a correct answer, whereas a heuristic technique does not. For example, a deterministic technique would never fail in finding a stock price on a Webpage, even if the table layout changed. A heuristic technique would fail since it, by definition, is only a best guess. Often that is enough, but don't bet your life on it.

Moving content from one Website to another is difficult to do if there are no published definitions, as there are for XML-tagged data. What if you want to combine stock prices from source A with news data from source B in some new way? To accomplish this task with today's data producers, you would need to parse the HTML pages and hope that the layout of those pages doesn't change. This is an ad hoc solution best left to experts that can worry about the capricious nature of Websites. An alternative is to work with Websites that provide data independent of data formats. In this example, we would take data from an established Website, parse it in realtime with either Tcl or Python, and generate XML data by issuing the following URL:

http://www.brazilhandlers.com:9090/Applications/ScriptingInterfaces/Tcl/getStocks.cgi?symbols=tern

The Brazil Web server implements a Tcl script to query a remote Website that assumes it is delivering data to a browser. Instead, the Tcl program takes the data, parses it, and places the results into the request property, which makes the data available to the BSL scripting facilities.

1 2 Page 1
Page 1 of 2