Jazkarta Blog

Plone+Salesforce: What’s Next?

2013-05-10T22:00:20Z

A big thank you to the Salesforce Foundation for inviting me to contribute a post on Plone+Salesforce integration to their blog. It was fun to reacquaint myself with all the tools and learn what’s new. Timely too, since we have just started work on a new Plone site for The Mountaineers which will include integration of all their custom content with Salesforce.com. We are working on this project in partnership with Percolator Consulting. They are engagement strategists and Salesforce specialists and are creating the CRM back end while we are creating the Plone site.

In the process of writing and discussing this post, I discovered that David Glick, who is working on The Mountaineers project with us, and Jason Lantz, from the Innocence Project, were having similar thoughts about the Plone+Salesforce toolkit. This week we had a chat to explore ways in which we might collaborate.

There are two main areas in need of improvement.

Salesforce.com API integration – The current Python library used by Plone is Beatbox, which is a wrapper for the Salesforce.com SOAP API. We now have Simple Salesforce, a Python library providing a low level interface to the Salesforce.com REST API (it returns a dictionary of the API’s JSON response). There is also a new socket-based streaming API. David and Jason are evaluating replacing Beatbox with one of these more modern components.
Message queuing – The current set of Plone+Salesforce tools rely on real-time communication with Salesforce.com, which slows performance and does not recover gracefully from network errors. A message queuing approach is sorely needed, where API requests can be queued for delivery and automatically retried. David and Jason are evaluating our options for Python message queuing. This includes but is not limited to plone.app.async. We would like the Plone implementation to build on a pure Python layer that could be used in any Python web framework (such as Django or Pyramid).

These items are critical to the success of The Mountaineers’ and the Innocence Project’s Plone+Salesforce integrations. In both cases the volume of API calls will be great enough that asynchronous communication is a must-have. Over the next few weeks we hope to create a plan that our two projects can collaborate on, for drastically improving the Plone+Salesforce infrastructure. This will benefit the entire Plone+Salesforce ecosystem – for example, the message queuing system could be used as the basis for a form-to-Salesforce solution that is more responsive and robust.

We would also like to improve the administrative tools that are available. Site admins will need a control panel that allows them to monitor and retry messages in the queue. Down the road it would be great to provide a way for integrators to define Plone to Salesforce content mappings through the web. Today it is necessary to do Python development to implement anything other than a fairly simple Plone+Salesforce web-to-lead form.

Our goal is to flesh out a plan by June 5th, when David, Jason and I will all be at Plone Symposium Midwest. Please contact me if you’d like to join our conversations, and maybe even sprint on an improved Plone+Salesforce infrastructure!

Responsive Design Made Easy with Plone

2012-12-20T22:02:51Z

Last week Marianne Kay tweeted that the Dumbarton Oaks site was a “beautiful example of responsive design”. (Thanks Marianne!) Kudos for the beautiful Dumbarton Oaks graphic design go to the design firm Might & Main. But kudos for the responsive behavior go to Plone and the Plone community, which made implementation easy. Here’s how we did it.

A responsive design provides an optimal viewing experience across a wide range of devices, from large desktop monitors to mobile phones. It uses CSS3 media queries, fluid, proportion-based grids, and flexible images to adapt the layout to the browser width. Normally, adding responsiveness to a website’s design means multiplying theming time by a factor of 2 or 3. But for the Dumbarton Oaks site responsiveness didn’t add much time because we used Plone’s built-in Diazo theming tool and started with a theme that provided responsive behavior out of the box. We didn’t have to write the code that (for example) makes images re-size, portlets change position, and menu tabs transform into drop downs for smaller screens. The Plone UI/CSS integration was already done for us.

Diazo is a fantastic tool that lets you map dynamic web page content from any CMS or web application into an HTML design. The mapping is done via a set of rules that define how to move elements from the content page into placeholders in the theme. Plone provides nicely integrated Diazo theming, soon to get even better with a through-the-web theme editor that will come with Plone 4.3. You can see a preview of it in this screencast made by Eric Steele.

Here is our recommended Diazo theming process.

Choose a Diazo Plone theme that provides the desired responsive behavior. For the Dumbarton Oaks project we used beyondskins.responsive by Simples Consultoria.
Create a static theme package to contain the designer’s HTML templates (including header and footer code), CSS, Javascript, and graphic assets. Diazo themes are pure HTML/CSS, so there is no need for designers to learn a special templating language.
Have the designer base his or her HTML templates on the ones provided in the responsive theme.
Add CSS files to the static theme that provide the responsive UI (from the responsive theme package), the Plone editing interface, IE fixes, and any special features provided by add-ons that are used on the site. Extract the contents of these files from the CSS that Plone serves with all products installed. Plone’s large CSS payload can be significantly pruned, only provide what’s needed.
To style the HTML, have the designer supplement this CSS in a way that doesn’t interfere with necessary UI elements or harm the responsiveness. It is helpful to add this graphic-design-specific CSS in a separate file.
Apply the static theme to the Plone site with Diazo rules.
Plone’s markup consistency and quality is a huge benefit during this process. Just give the designer sample HTML and let them do what’s needed in the theme’s CSS.

When we engaged Might & Main to do the Dumbarton Oaks design, we forgot to tell them it was going to be responsive. They were very perturbed when they found out, since they had not budgeted to do the extra work that is normally required. We told them not to worry because that extra work had already been done in beyondskins.responsive. And sure enough, Dumbarton Oaks got a responsive design with very little effort from Might & Main.

Here is what the site looks like without the Diazo theme applied.

Here is what the themed site looks like on the desktop.

And here is what it looks like on a mobile device.

Plone-Salesforce Integration Just Got More Powerful

2012-10-05T03:30:47Z

We recently helped the Innocence Project finish collective.salesforce.fundraising, an open source fundraising package designed to help non-profit organizations raise money online. The system is designed to function either as a standalone donation system or to be integrated into a larger Plone website. The Innocence Project is currently using it as a standalone donation system.

collective.salesforce.fundraising

The brainchild of the Innocence Project’s Jason Lantz, who developed the package with help from Groundwire and Jazkarta, this plugin adds significantly to the Salesforce.com functionality that can be tapped from the Plone content management system. The Plone CMS/Salesforce.com CRM integration capabilities previously consisted of:

Submitting form information from a Plone form into Salesforce.com (web-to-lead forms)
Adding a new lead to a Salesforce.com account when a new Paypal payment is made
Making Events in your Plone site “RSVP-able” with data saved directly to Salesforce.com campaigns
Authenticating in Plone with account information (login, password and properties) which is from Salesforce.com
Dynamically displaying Salesforce.com data in Plone

That’s already a darn good package, but collective.salesforce.fundraising adds features that take things to a whole new level:

The ability to take one time and recurring donations through an uncluttered, modern UI. All donations are sent directly to the payment processor (Authorize.net or Recurly), no credit card data is stored or transmitted.
Integration with Salesforce.com campaigns. They can be created in Salesforce.com or Plone, and all changes and donations made in Plone are added to the Salesforce.com campaign.
The ability to create campaign timelines and goals. Progress indicators in the UI show progress towards the goal and the time remaining.
Personal fundraising, which allows users to create personal campaign pages for any campaign. Users can set a goal and promote their campaign to their friends; all donations roll up to the parent campaign in Salesforce.
Social integration via Janrain, a social login and sharing service. This is important for personal fundraisers, since it allows login through a variety of social accounts, and sharing on multiple social networks to help spread the word about personal campaigns.

And there are many smaller features, like memorial donations, donor testimonials, fundraising “seals” providing third party analysis and endorsements, and an easy way for users to share campaigns with their social networks via “share messages” which can be tracked for effectiveness.

How Jazkarta Helped

The Innocence Project engaged Jazkarta to help with the donation and personal fundraising capabilities. This included selling tickets to events, adding campaign landing pages and personal fundraising pages, and working out some of the details of the social login process. Personal fundraising is a compelling addition to the package. The ability it provides for individuals to set themselves up as fundraisers under an umbrella project puts collective.safesforce.fundraising head and shoulders above many of the other solutions available to non-profits.

Convio Common Ground Customers Take Note

The addition of personal fundraising to collective.salesforce.fundraising comes at a particularly opportune moment. Only days after Jazkarta’s work was done, Blackbaud announced it was discontinuing Convio Common Ground, a fundraising and donor management application built on top of Salesforce.com and used by many small to mid-sized non-profits. A Plone+Salesforce.com powered fundraising site offers a great option for the 400 or so Common Ground clients that have to make a decision and migrate to a new system within the next 19 months. The Plone CMS and all of the plugins mentioned in this post are open source software with no licensing fees, and Salesforce.com is a great deal for small to mid-size non-profits. Through the Power of Us program, the Salesforce Foundation will donate 10 enterprise licenses worth $1600/month to 501(c)3 organizations, and any additional licenses are 80% off. There is also a 50% discount for in-person training and the customer portal is priced at $24/user/year.

Contact Jazkarta if you’d like more information about Plone-Salesforce powered online fundraising solutions.

Go Botany!

2012-07-16T18:55:29Z

We’ve been having classic summer weather here in New England – gardens, fields and forests are exploding with lush vegetation. It brings out my inner botanist – something I actually have advanced training in. These days botany is usually a hobby for me, but two years ago my Jazkarta world and my botanist world came together on an amazing project – Go Botany.

In 2009 the New England Wild Flower Society was awarded a major grant from the National Science Foundation to develop an innovative suite of tools to teach botany and plant identification. This multi-year project has many deliverables, but the overarching goal is to develop radical new web-based tools for identifying plants that will improve botanical knowledge and science education for novices and citizen scientists. In other words, get more people hooked on plants!

Discovery

NEWFS selected Jazkarta to be their technology partner and we worked on the project throughout 2010. We started with a discovery phase, during which we sorted out the various tools, prioritized requirements, defined the technology platform, and created a plan for tackling development. Our plan relied on an agile, iterative approach; the project owner – Elizabeth Farnsworth, NEWFS Senior Research Ecologist and PI on the NSF grant – was always in charge of what we delivered next. This was essential since at the beginning of the project no one had a very clear idea of what the tools should look like. By working in an agile way we were able to easily adjust the plan as we learned more. This quote from the book The Art of Agile Development, by James Shore and Shane Warden (O’Reilly) summarizes the situation nicely.

The beginning of every software project is when you know the least about what will make the software valuable. You might know a lot about its value, but you will always know more after you talk with stakeholders, show them demos, and conduct actual releases. As you continue, you will discover that some of your initial opinions about value were incorrect. No plan is perfect, but if you change your plan to reflect what you’ve learned – if you adapt – you create more value.

Technology Platform

Python was a natural choice for implementation language because NEWFS IT staff were already using it. In addition to being mature, powerful, secure, open source, and object oriented, Python has a wide array of libraries available for scientific computing and GIS.

We considered several Python web frameworks that were popular at the time. Plone (the CMS in use at NEWFS) is not well suited to building the sort of custom applications we needed, but some of the other frameworks (like Grok and repoze.bfg) didn’t have enough functionality. We chose Django, which had gained traction in the Python community because of its simplicity, admin features, form technology, and the availability of numerous add-ons. Two of those add-ons were particularly critical to the success of the project: GeoDjango for adding GIS features, and Pinax for adding social networking features.

We had a lengthy in-house debate over the choice of database technology. PostGIS (the spatially enabled version of PostgreSQL, an industrial-strength open source relational database) was the clear winner for its compatibility with the Django ORM, and it was required for GeoDjango. But the data model for the botanical information was essentially a star schema (one table in the middle looking up information in dozens of auxiliary tables for attributes and vocabularies), which is much more efficient to implement in an object database than in a relational database. In the end, we chose to use PostgreSQL/PostGIS for everything, mostly because we knew it would make life easier for future developers and sysadmins on the project.

We needed one more component to implement the Go Botany tools, which were going to rely heavily on various types of search. We chose Solr, an open source search server based on the Lucene Java search library, to provide indexing and search services for all tools. It provides cross database search functionality over a REST interface, including features like custom query structures, common document type extraction, geographic searches, and search facets.

Implementation

By May we had put together a team and launched into release planning and implementation. The team consisted of Jazkarta developers and NEWFS developers working together on the project, with me as the PM and Elizabeth as the project owner. We like to have joint, collaborative teams with our clients whenever we can, it’s a great way to get their IT staff up to speed quickly. We also spent some time teaching Elizabeth about agile development practices, which helped her do a great job supplying us with the information we needed, when we needed it (user stories, feedback, etc.)

Database and API

Our first project was to design and implement the PostgreSQL database of botanical information that would drive all the Go Botany tools. NEWFS had previously built a Microsoft Access-based tool for entering the copious amounts of botanical data required for the project – dozens of characteristics for each of the thousands of species of New England plants. A time consuming part of building the database involved automating the import of all the MS Access data. We also implemented an API wrapper around the database that was optimized for the kinds of queries and transactions that would be required. The API calls were largely implemented as json web services for ease of integration into Javascript user interfaces.

The database plus API was the most technologically challenging part of the project. It had to provide a flexible and fast foundation for the entire family of Go Botany tools. It was also interesting to write “user stories” for this part of the project – the “users” were all the Go Botany tools, which would need to get information from the API.

Simple Key

Once the database and API foundation was laid, we began work on the Simple Key – an interactive, online guide to identifying 1,200 of the more common New England plants. This was the most challenging part of the project from a user interface point of view. It needed to be fun and exciting to seduce novices into learning about plants and botany, while at the same time being useful to pros. NEWFS engaged user experience designer Matt Belge (Visionlogic.com) to help them define the tool we would be building. Based on lots of interviews and other research, Matt produced wireframes for successive bits of Simple Key functionality, and we figured out how to implement them.

The result is unlike any previous plant identification tool. The Simple Key presents the user with a series of questions about their plant’s characteristics that are designed to home in on the species identification as efficiently as possible. It does this based on the questions that have already been answered and the features that the user is able to observe. The secret behind this behavior is an information gain algorithm that generates optimal decision trees.

When the user has identified their plant, they can go to a species page with a wealth of information – including maps of its geographic range, diagnostic characteristics, memorable facts, and gorgeous color photographs of the plant and its leaves, flowers, bark, etc. in different seasons. A fitting end to a successful quest.

And It’s Open Source

Because taxpayer dollars through the National Science Foundation supported this project, the software and data on plant characteristics will be open sourced. Details are still being worked out, but the goal is to enable others to contribute and improve the code over the long-term. Stay tuned to the Go Botany site for an announcement of the details. NEWFS staff are also engaged in adapting the database to allow other botanical institutions to customize the data for their particular region.

But This Is 2012!

Yes, all of our Go Botany work was done in 2010, so why am I blogging about this now?

We got Go Botany tool development off to a good start, but a lot of work remained after we were done. Seven botanical data specialists did a massive amount of manual data entry: scores of characteristics for thousands of plant species. Four image collectors scoured the web, publishers, and individual contacts for the gorgeous photographs used on the site. Botanists developed technical descriptions of the New England plant families and genera. Many NEWFS staff created videos, help pages, an illustrated glossary of terms, and other content. A web design firm, Fresh Tilled Soil, was hired to create the graphic design for the Go Botany website, including the Simple Key. NEWFS developers applied the theme to the Django site, as well as continuing to refine and fix the Python and Javascript code.

All of this hard work came to fruition in April: the Go Botany website is now live at http://gobotany.newenglandwild.org/!

So if you live in the northeast and you sometimes wonder what the name of a plant is, please try it out! Go Botany works best with Firefox, Safari, and Chrome on iPads, tablet and desktop computers (and the UI is being adapted for phones) so you can take it with you on walks and have an expert assistant for your botanizing. If you have comments, NEWFS would love to get your feedback at gobotany@newenglandwild.org.

Templer and ZopeSkel

2012-07-12T16:37:32Z

At the sprint following Plone Symposium East, I helped Cris Ewing write documentation for Templer, a Python code generation system that was derived from the older ZopeSkel application. I knew very little about Templer before the sprint, but the experience taught me that it is an incredibly useful tool for Python developers working with Plone, Django, Pyramid, or any other Python framework. Whether you are a freelancer or part of a larger web development organization, Templer can make you more productive.

Templer is a general-purpose system for generating code skeletons from pre-defined templates. End users provide information through an interactive interface, which is used to generate a skeleton of files and folders that make up a Python project. The projects that can be built with Templer today are:

Python namespace and nested namespace packages
Buildouts and recipes to extend the zc.buildout system
Namespace and nested namespace packages for Zope
Add-on packages for the Plone CMS

There are also commands (based on Python Paste) for adding features to Plone software projects that were created with Templer, such as browser views and Archetypes content types.

The functionality that Templer provides is very handy, but the thing that makes it a Python developer’s secret weapon is the fact that it can be extended and used to build applications focused on particular problems. ZopeSkel, which provides a suite of templates for generating Zope and Plone projects (buildouts, themes, add-ons, etc.), is one example of a Templer application. During the sprint, Ian Anderson began work on templer.django, a new Templer package to provide templates for Django projects. This package could form the basis of DjangoSkel, an application to quick-start Django projects. If you are in the business of creating Plone or Django websites, you can quick-start your own projects with a Templar based application. For example, MyPloneSkel could set up your production and development buildout configs just the way you like them, plus optional packages for Generic Setup policies, a Diazo or standard theme, and custom content types.

Thanks to everyone who has worked on ZopeSkel and Templer over the years to make such a useful suite of tools.

Kale Will Save the World

2012-07-12T22:31:17Z

I have to hand it to the University of Minnesota Press, they do a great job keeping their homepage fresh. It has lots of portlets around the edges, and a big slot in the middle to hold a featured image. Every few weeks they swap a new image into that slot, and voilà! new homepage. This week’s UMP homepage features kale. Really. It made me both laugh and click (which took me to a page featuring local food writers and their books.) Their first hompage featured rock music (captured in the client blurb on our website), and by the time you read this it will probably be featuring something else. But whatever is featured, you can bet it will be intriguing and will draw you into exploring UMP’s amazing catalog of books.

Kudos to our partners, Might & Main, for a compelling design (it even looks good on mobile devices) that can be refocused with a single image upload. And kudos to the Plone CMS for making this so easy to implement. We used ContentWellPortlets to provide the bottom porlet slots. The central image goes into the homepage body, and the rest of the content comes from normal Plone text, RSS, and collection portlets. The site is themed using Diazo (via plone.app.theming), but the homepage magic is mostly CSS. The various portlets are styled depending on their location (right, left, bottom), plus there is extra CSS for the Twitter and New Releases portlets. This tyling is made possible by the fact that Plone provides CSS identifiers for just about everything on the page.

The result? A homepage that keeps surprising me.

Managing Agile Plone Projects

2012-03-01T17:50:30Z

If you liked the post I wrote last month about agile development with Plone, you might be interested in an upcoming class called Managing Agile Plone Projects that I’m teaching at Plone Symposium East. Consider this scenario:

All the pieces have fallen into place for that big website project. You’ve chosen Plone, you’ve lined up the budget, you’ve gathered a team of developers, you’ve selected a design firm. You’ve worked with all the stakeholders and come up with a long list of features. Plone and its add-ons will provide many of them, but some will need custom development. How are you going to manage the project? You have a part time team and the project is part design. part configuration, and part development – how can you use agile techniques?

If you find yourself in this situation, consider joining me at Penn State on May 21st. I’ll use a combination of lectures, discussions and group exercises to share the approach I’ve developed to manage this type of project. You can read more details and register here.

Symposium East is always a great conference and a great time. If managing projects is not your thing, there are other excellent classes that may interest you:

Theming Plone 4 with Chrissy Wainwright
Creating Plone Content Types with Dexterity with Steve McMahon
Getting off the Ground with Plone with Cris Ewing
Python Fundamentals with Chris Calloway

I hope to see you there!

Agile Development with Plone, Revisited

2012-01-20T18:39:54Z

The announcement of the 2012 Plone Symposium East (thank you Weblion! always a great gathering) is reminding me of the talk I gave there last year on Agile Development with Plone. Here at Jazkarta we’re engaged in another large project, similar in many ways to the one that inspired that talk. Because our project management tools and techniques have evolved a bit since last year, I figured it was time for an update.

The Projects

Last year at this time, we were just getting started on a major website redesign for the University of Minnesota Press. There’s lots of information about this project in my Plone Conference presentation How To Get a Fabulous Website on a Modest Budget Using Plone. Briefly, it involved a major data conversion from a FileMaker database into Plone, a small number of complex custom content types, extensive design work, and a featured “explore” section powered by eea.facetednavigation.

Our new project is a major website redesign for Dumbarton Oaks Research Library and Collection, a research institute of Harvard University and the home of a little known but outstanding museum and garden in Washington, D.C. We began with a discovery phase last summer, and have been working on implementation since November.

What’s the Same

Both of these are fixed budget projects with a large number of desired features, and at the outset many of these were inadequately defined for development purposes. In both cases there are multiple stakeholders with differing priorities, but there is an intelligent, proactive project owner who is trusted and empowered to make decisions. The Jazkarta team is the same – Carlos de la Guardia in Mexico, Alec Mitchell in California, and me in Massachusetts – and none of us ever work full time on any of our projects.

This all adds up to the same basic project management challenge: how to adapt agile techniques to manage a fixed price, flexible scope project with a distributed, part time team.

What’s Different

Happily, the Dumbarton Oaks project does not involve a complicated data import from a FileMaker database. For UMP this required a collective.transmogrifier script, many many many trial and error imports, and over 60 hours of Jazkarta development time. Dumbarton Oaks currently has a static HTML website, which is much simpler to import. Funnelweb made this easy – only a few trial imports and some content massaging were required. We’re not a quarter through the project but all the old content is already on staging. Total development effort: 10 hours.

Because the content import was relatively easy and the project mostly requires custom features and content types that are independent of each other, we have been able to divide up the work into several releases. This gives us a series of smaller planning horizons and allows us to be more agile (more on this later). Each release can potentially be deployed to production – and even if we decide not to, new content is developed on staging which gives the new features a thorough vetting. No matter how much you thought you tested, there are always problems that don’t get discovered until the software sees serious use. Having a series of small production deployments avoids piling all the bug fixes and theme tweaks into a single mega deployment phase at the end of the project.

Our Evolving Agile Development Practices

Our basic development process remains the same:

Discovery
Define schedule and resources (how the budget will get spent)
User story development
Story estimation (we use planning poker)
A series of development iterations alternated with evaluation periods
Production deployment

During discovery the project owner and I distributed the feature requirements into a series of releases, based on a combination of priorities and keeping all the stakeholders happy. (Everyone got something in almost every release.) This high level release plan allows us to do our story development and estimation one release at a time, which saves time and money – especially considering that all the developers participate in planning poker sessions. Since we know the customer wants more than their fixed budget will get them, and we also know they will change their mind about what they want as the project progresses, it makes sense to postpone the effort of writing and estimating stories we may never get to.

As in previous projects, I took the first cut at turning the project requirements into stories, then discussed and revised the stories with the project owner. Because I have intimate knowledge of Plone, I can translate requirements into stories that make sense to develop. For example, we don’t need to spend time defining stories for CMS functionality or for things that Plone has great add-ons for (like staff directories and bibliographies). Some things may turn out to need tweaking, but it is better to give the customer stock Plone and add-on features to try out first. They may decide to live with the stock features and put their development dollars into new features instead of making changes.

Our communication tools remain the same: Skype, IRC, chat, Google Calendar, and Google Documents. This combination is the secret sauce that allows a distributed team to work as if they were in the same room. We did switch from ClueMapper (based on svn and trac) to github for source code repository and ticket tracking. We miss a few management features, but the developers are much happier working in a dvcs.

Our development iterations remain the same – one week each, followed by a week of evaluation during which the customer has time to test and reflect on what they want to do next. We kick off each iteration with a planning meeting, which surfaces any questions about the stories we’re tackling and allows us to do task breakdown and assignment while we are all in the virtual room of a Skype conference call. During the iteration we have a short standup meeting every day to keep things on track – everyone says what they did yesterday, what they’ll do today, and raises any blocking issues. My notes from the standups are written in a Google spreadsheet which is shared with everyone. This gives the project owner access to detailed information about how things are going, if they miss a standup meeting.

The biggest change we’ve made for this project is to adopt ScrumDo for story development and iteration planning. ScrumDo is an open source agile project management tool that’s written in Python. We are using the hosted, SaaS version that’s at scrumdo.com – accounts are free for up to 10 users and 10 projects. ScrumDo has a friendly drag and drop UI that makes it easy for for project owners to plan iterations, edit stories, add comments, and see results. Unlike Pivotal Tracker, it allows flexible iteration scheduling and story and task assignments. Google Docs provide better simultaneous editing for multiple users and more flexible formatting, but ScrumDo makes it really easy to add and change tasks, assign them, and mark them as complete. If it managed estimated and actual hours I’d totally love it.

ScrumDo added a Planning Poker tool last March, and this has been a real time saver. It’s a web page where the scrum master selects a story to size and everyone puts down a story point card. Once you play your card you can see the others, and after discussion and reaching consensus, the scrum master assigns the winning value to the story. What set of cards to use is one of the ScrumDo project settings (we use modified Fibonacci).

ScrumDo Planning Poker interface

The Value of Agile Project Management

The planning and standup meetings required by an agile approach do take time. For the UMP project, the meetings and project management activities combined used almost 40% of the total budget. But what is the alternative? Written requirements “thrown over the wall” to developers are a recipe for failure. The customers and the developers have to talk to each other to understand the project and its development trade offs. I’ve never had a customer complain that I’m using too much project management time when I’m running a project in this way. The value of open and frequent communications and iterative development cycles is very clear to them. They get full ownership of the project, choose exactly what it will deliver at every step along the way, and in the end they are delighted to get the system they envisioned.

Plone Quick Start for EC2 now available in all regions

2011-01-20T08:25:18Z

Our Plone Quick Start Amazon EC2 machine images have been in enough demand to warrant availability in all four EC2 regions. We have responded to that demand and now anyone can test drive Plone 4 easily in their preferred region. Just go to our free signup page and fill out the form. You will receive a confirmation email and will be able to create your instance immediately after that.

The Plone 4 Quick Start gives you a fully working Plone 4.0.2 site, running on Ubuntu 10.10. The site has two instances in a high availability configuration using Nginx as an HTTP frontend, Varnish for caching and HAProxy for load balancing. Supervisor is used for starting and stopping the Plone instances and ZEO.

If you’ve been wanting to try out Plone or know someone who is, this is probably as easy as it gets. Give Plone a chance.

Plone on WSGI sprint report

2010-11-22T21:45:11Z

Two laptops, two sprinters and two bottles of whiskey. What could possibly go wrong?

This is my attempt to document what I can only describe as the “Plone on Whisk-gi” sprint… As part of the Plone deployment sprint at the recent Plone Conference in Bristol, Reed O’Brien and I decided to try to get Plone 4 working on WSGI.

Why is this important? Hanno Schlichting (current Zope2 release manager) tells us on his blog post about the recent Zope 2.13 release:

Zope 2.13 comes with native WSGI support. First pioneered in the repoze.zope2 project, this capability finally found its way back into the core and obsoletes the externally managed project. With WSGI, Zope 2 can natively talk to a variety of web servers and isn’t restricted to its own ZServer anymore. It also opens up new possibilities for writing or reusing middleware in Zope 2 or factoring out capabilities into WSGI endware. It’s expected that this new deployment model will over time become the default and the old ZServer implementation will be deprecated. There’s no concrete timeline for this yet.

What is Hanno saying here? Basically, it means that Zope 2.13 (and subsequently Plone) can run on a server without the added complexity of a long-running process (ZServer). Just as PHP-based applications enjoy ease of deployment since Apache already has mod_php enabled, so too could Zope-based applications be easily deployed if mod_wsgi were enabled.

In thinking about how Plone and Zope can be more accessible to systems administrators and hosting companies, moving to a scenario in which Zope does not require a separate long-running process, and can be deployed essentially at the Apache layer is a step in the right direction.

As Hanno alludes to above, with the WSGI deployment model, Zope/Plone are able to tap into a whole ecosystem of 3rd party WSGI middleware tools. Repoze is an initiative to extract many of the core Zope technologies into reusable WSGI middleware components, that can be used in other Python web application frameworks. More WSGI components can be found on PyPi by searching for “wsgi”.

Also, by adopting a more typical deployment method, Zope will not seem so exotic to developers coming to Zope from other Python web frameworks. For example, the official Django documentation recommends deploying Django using mod_wsgi, so Django developers will likely already be familiar with this way of deploying Python web applications. Incidentally, if you’re doing Django development, check out twod.wsgi which makes WSGI a first-class citizen in Django applications.

How does Plone fit in this picture?

Plone, arguably being the most popular Zope2 application on the market, can stand to benefit from the WSGI deployment strategy. There have already been some efforts to do so with repoze.plone and repoze.zope2 (now deprecated with the release of Zope 2.13).

Calvin Hendrix Parker blogged about deploying Plone and Zine together with Deliverance using Repoze, and Wojciech Lichota blogged about his buildout for Plone 3 with Deliverance on WSGI. Nathan Van Gheem has also written a couple blog posts about his experiences running Plone 4 with a Zope2 WSGI and running Plone 4b4 with Zope 2.13.0a1.

Now that Plone 4 is officially out and Zope 2.13 has had a final release, the instructions for getting this working are considerably easier.

Here are the steps required to get Plone 4 running under WSGI using paster.

1. Create a buildout.cfg file with the following:

[buildout]
extends =

http://dist.plone.org/release/4.0.1/versions.cfg


http://download.zope.org/zopetoolkit/index/1.0/zopeapp-versions.cfg


http://download.zope.org/Zope2/index/2.13.0/versions.cfg

parts =
    instance
    paster

versions = versions

eggs +=
    roman
    Products.ZSQLMethods

[versions]
Zope2=2.13.0
Products.ZSQLMethods = 2.13.3

[instance]
recipe = plone.recipe.zope2instance
user = admin:admin
eggs +=
    ${buildout:eggs}
    Plone
    Paste
    PasteScript
    PasteDeploy
    repoze.tm2
    repoze.retry

[paster]
recipe = repoze.recipe.egg
scripts = paster
eggs = ${instance:eggs}

2. Create a zope2.ini file:

[DEFAULT]
debug = True

[app:zope]
use = egg:Zope2#main
zope_conf = %(here)s/parts/instance/etc/zope.conf

[pipeline:main]
pipeline =
    egg:paste#evalerror
    egg:repoze.retry#retry
    egg:repoze.tm2#tm
    zope

[server:main]
use = egg:paste#http
host = localhost
port = 8080

3. Run the buildout and modify the zope.conf file:

Run the buildout which will generate the paster script in the bin dir and the zope.conf file in parts/instance/etc. You then need to comment out the <http-server> part in this file. If you don’t comment out these lines, then paster will complain that there is already a server running on port 8080.

#<http-server>
# valid keys are "address" and "force-connection-close"
#  address 8080#
#</http-server>

4. Start up paster using the zope2.ini file:

$ paster serve zope2.ini
Starting server in PID 4666.
serving on http://127.0.0.1:8080

You can now go to http://127.0.0.1:8080 to see your Plone site served up by Zope on WSGI!

In the next blog post, we will describe how to get this semi-working in a production environment using mod_wsgi. For a sneak preview you can check out this buildout that Reed put together at the sprint. Note that there are a few patches that must be applied to Zope2, and hopefully Tres Seaver is reading this and will respond to them.

What’s the official word on WSGI support for Plone 4?

Plone 4.1 will ship with Zope 2.13, but as Hanno states in PLIP #10776:

While Zope 2.13 provides native WSGI functionality, it is outside of this PLIP to make use of this in any way. There’s a large number of open questions around WSGI and the changes it requires to instance creation and setup, which warrant their own PLIP. A good number of those will also have an impact on the Zope 2 codebase itself, which is going to be already past a beta release by the time this PLIP gets started and thus will have to wait for Zope 2.14. I expect to see more community experimentation being done with WSGI once the capability is there. We might see a PLIP for Plone 4.2 to see those experimentations being solidified into good practice, standards and documentation.

So in other words, it’s still on the bleeding edge to try to use it, and probably not advised for production use just yet. As Hanno mentions, it’s expected that there will be more experimentation around WSGI, and as a community, we can come up with some best practices and documentation for how to do it in a sane way. Please let me know if you have done any experimentation with WSGI already, or if you’re interested in getting involved in these efforts.