Serve clients' specific protocol requirements with Brazil, Part 5

Manage users and content with Brazil

1 2 3 Page 2
Page 2 of 3

Some Websites might have legal restrictions on the reuse of their content, so make sure you don't violate a site's copyright agreement. Tampering with Web content causes many concerns other than legal ones. For example, a filter could insert ads or, even worse, change the site's content without your knowledge or permission. For more detail on this issue, see "Proposed Web Protocol Sparks Tampering Fears." Like all technologies, Brazil can be used inappropriately and illegally when in the wrong hands.

For our purposes, the pollnews handler configured in the config file will periodically call the GetNews.html page to acquire news about the stocks we have interest in. I used Jython to parse the news data because I prefer Jython to JavaScript and Perl. Brazil technology makes information available to Jython by providing access to the fundamental objects Request and Server. A Jython developer can get and set these values for properties using the methods getProperty() and set(). Take a look at how this works with a small but powerful Jython example, GetNews.py, that scrapes the data and turns it into a more malleable form:

# Have the Brazil Server run the Jython interpreter on the following code.
<server language=python>
# Packages we will need to use 
import re, string
from sunlabs import brazil
from java import io
# Since these stocks are common for all users they are in the server 
object.
stocklist = server.props.getProperty("stocklist")
# Convert the returned string to a list
stocklist = string.split(stocklist, " " )
# Iterate over the list, creating a url for each stock.  Use the Brazil
# HttpRequest class to send a request for contents at the url.  Then
# convert the response to a string, select out the content (i.e. news
# articles) we want by matching against a regular expression, and then
# reverse the order of the results.  Finally, for each news line, select
# the items we want and save them in server.props.
for stock in stocklist:
  url = "http://finance.yahoo.com/q?d=v3&o=t&s=" + stock
  news = brazil.util.http.HttpRequest(url)
  # news.setProxy ("webcache.east", 8080)
  bos = io.ByteArrayOutputStream()
  news.getInputStream().copyTo(bos)
  s = bos.toString()
  # Create a regular expression to process the data.  Note, it will take
  # time to test and develop for different Websites.
  prog = re.compile(r"\s+(\w\w\w)\s+(\w\w\w)\s+(\d+)\s+(\w+)\s+"
                    r"<a.*?href=(.*?)>(.*?)</a>", re.S)
  # Apply the regular expression and reverse the order of the results
  m = prog.findall(s)
  m.reverse()
  # Finally iterate over the news items and extract and save the data we 
want
  index = 0
  if m != None:
    for x in m:
      index = index + 1
      text = x[3] + ".news.text." + `index`
      url = x[3] + ".news.url." + `index`
      server.props.put(url, string.replace(x[4],"\n",""))
      server.props.put(text, x[5])
</server>

User interface

When the user goes to http://www.digiorgio.com:9001/, the index.html file is processed. This page presents a stock portfolio to the user, along with the option of adding or removing stocks. The HTML file below, index.html, allows users to add stocks to their portfolios and display those portfolios. The page uses BSL to extract values from the server. When a user requests a page, the server processes the page and substitutes in the values of the requested properties.

On the server, Brazil can process any language that has a handler; handlers have been written for Tcl and Jython. The tag <server language=python> instructs the Brazil server to treat the ensuing text as Jython source code. The PythonHandler interfaces to the Brazil server via side effects: the setting and unsetting of property values. The HTML page below also includes some BSL, which I discussed in prior examples. Though three distinct languages appear on the page -- Jython, BSL, and HTML -- the power of expression is worth any potential confusion. index.html has numerous comments explaining how BSL, Jython, and the Brazil technology interact, thereby allowing you to rapidly prototype the application in less than 200 lines of code:

<html>
<head>
<title> Building Dynamic Content from existing Web 
Content</title>
</head>
<body>
<LINK REL=stylesheet TYPE="text/css" HREF=/jw.css>
<!-- When a form is posted the values of the form are prefixed with 
query.
As specified in the config file. One of the values is a hidden
variable called submit. If this value exists then we know the user has
clicked the submit button.
-->
<if query.submit>
<server language=python>
import string
# The HTML multiple select option sends all parameters with same name.  So
# we won't be able to use the query property of the PropsTemplate to 
# separate them for us, thus we need to parse the query string and 
generate 
# a list of the user's selected stocks
newstocks = request.props.getProperty("headers.query")
# newstocks will contain a string like this: 
my.stocks=SUNW&my.stocks=IBM&...
newstocks = string.split(newstocks, "&")
# Make newstocks a list
mystocks=""
# If the value starts with my.stock then use it.
# There could be other parameters not related to my.stocks
for s in newstocks:
    if s[:3] == "my.":
        mystocks = mystocks + s[string.find(s,"=") +1:] + " "
# Set the stocks in the request props, not in server properties since these
# are of interest to each individual user.
request.props.put("my.stocks", mystocks)
</server>
<!-- The SetTemplate causes the value of my stocks to be put into the
     session properties, which in our case are persistent and written to
     disk periodically. This is not an ACID implementation. Interested
     users can replace the SetTemplate with an ACID instance like a
     database.
-->
<set name=my.stocks value=${my.stocks}>           
</if query.submit>
<!-- Display the user's portfolio. Note how table headings are not
     displayed if the user does not have a portfolio.
-->
<if not name=my.stocks value="">
  <h1>Your Portfolio</h1>
  <table cols=3 width="100%">
    <tr>
      <th width="10%" align=left>Stock</th>
      <th width="15%" align=left>Price</th>
      <th width="75%" align=left>News</th>
    </tr>
    <!-- Iterate over the users stocks and display them -->
    <foreach name=i list=${my.stocks}>
      <tr>
        <td align=left> <property name=i> </td>
        <!-- Get the price -->
        <td align=left> <property name=stock.${i}.price> 
</td>
        <!-- Get the first news item, there are many of them.
             The second news item would be news.text.2 and so on.
         -->
        <td align=left>
          <tag>a class="sitelink"
            href=<property name=${i}.news.url.1>
          </tag>
          <property name=${i}.news.text.1></a>
        </td>
      </tr>
    </foreach>
  </table>
<else>
  Please add some stocks to your portfolio
</if not name=my.stocks value="">
<center>
  Add or remove Stocks from your portfolio
  <form>
    <input type=hidden name=submit value=true>
    <!-- BSL can be used to generate complex forms very quickly without 
hard
         coding the values of the form. The form below is a multiple select
         with dynamic selection of the user's current selection. This is 
very
         common on the Web but I challenge you to find a simpler example of
         this using alternate technologies.
    -->
    <select name=my.stocks size=5 multiple>
      <foreach name=i list=${stocklist} sort>
        <set name=attr value="" namespace=local>           
        <foreach name=j list=${my.stocks}>
          <if name=j value=${i}>
            <set name=attr value="selected" 
namespace=local>           
          </if>
        </foreach>
        <tag>option <property name=attr> </tag> 
<property name=i>
      </foreach>
    </select>
    <br>
    <input type=submit value="Make current selection my portfolio">
  </form>
</center>
<!-- The code below lists all the server and current request properties.
     It is very useful for debugging.
-->
<if name=debugProperties>
  <hr>
  <h2>diagnostics</h2>
  <table border=2>
    <foreach name=item glob=* sort>
      <tr>
        <td><property item.name></td>
        <td><property item.value></td>
      </tr>
    </foreach>
  </table>
</if>
</body>
</html>

Summary

In this article, I showed how to configure Brazil to support multiple users with state. These users are presented personalized content synthesized dynamically from other Websites. Using regular expression and Jython, the data is heuristically processed to produce new content that BSL generates into HTML, which in turn the browser processes and displays to the user. As I demonstrated, you can accomplish that task with less than 200 lines of HTML code. In prior articles I discussed JRMS, the Wireless Application Protocol (WAP), Palm query application (PQA), and the Java 2 Platform, Micro Edition (J2ME); you should be able to easily add support for those environments as well. You'll find a commercial prototype using some of these concepts in Resources.

In Part 6, I will discuss how Brazil works with Jxta, a Sun research effort aimed at providing complete access to Web.

Recently, Sun Microsystems released an early version of the Java API for XML Messaging (JAXM), which provides Java support for sending and receiving SOAP messages with layered support for protocol-independent frameworks. I will provide a small example in my next column that demonstrates how to send and receive SOAP messages in a Brazil environment. The example also utilizes Jython technology to enhance server-side scripting with a more object-oriented scripting language.

Future applications

You can apply the Brazil technology to Websites that require HTTPS connections. For example, I like to keep my accounts at CitiBank, Allstate, Salomon Smith Barney, Yahoo!, Half.com, and so on. Instead of going to each Website for information, I'd rather integrate the content into one page. For example, when I look at my online checkbook, I could either have clickable URLs to the actual sites or my entire financial status displayed. That way, I wouldn't need to re-enter data in yet another portal, which is an increased security risk. You should be able to extrapolate the process I described in this article to put together your own personal portal similar to an online checkbook. The stock example grabs content from different Websites and creates a new page. This same technique could be applied to other Websites in most cases.

As Sun ONE and Web services develop, it will be interesting to see whether companies provide data to their customers in a more useful encoding format. Wouldn't it be nice if your bank and credit card companies provided you with a dynamic Web service that you could query in realtime for your bank account balance, so that you didn't have to set up an Excel spreadsheet?

An upcoming release of the Brazil technology will contain a more sophisticated example of managing users with accounts and portfolios. This release should be available at http://dev.experimentalstuff.com in late September.

Acknowledgements

I would like to thank the Brazil team, especially Steve Drach and Stephen Uhler, for their support with the example.

Rinaldo Di Giorgio writes the Java Developer column for JavaWorld.
Related:
1 2 3 Page 2
Page 2 of 3