Navigation:
Documentation
Archive



Page Tree:

Child pages
  • BIP 0.4 Bamboo Services Platform

This wiki space contains archival documentation of Project Bamboo, April 2008 - March 2013.

Skip to end of metadata
Go to start of metadata

WORK IN PROGRESS

This is an outline for Phase 1 (24 months) of a Bamboo Implementation Proposal.
The purpose of this document is to provide information to institutions and organizations participating in the Bamboo Planning Process so that they can help determine (1) the long term future of Bamboo and (2) define what activities Bamboo will carry out in its first implementation phase. The intent of this document is to solicit community input toward the ongoing development and revision of the implementation proposal. As this is an early draft, it is not yet a commitment to carry out all or any of this work.
Please note that we are updating this document frequently based on wide ranging input from the Bamboo community. These updates will occur periodically and will be indicated as ".1", ".2", ".3", etc updates. In addition, we will occasionally make major document revisions. These are noted as "1.X", "2.X", and so forth. Between major document revisions there may be some inconsistencies in language used between the sections of the document.

4.3 - Bamboo Services Platform

4.3.1 Summary

A Bamboo Services Platform ("Platform") is foundational technology necessary to support software services for arts and humanities scholarship. Bamboo Services will run (be deployed) on a Bamboo Services Platform, as web browsers and word processors run on an operating system (e.g., Microsoft Windows).

Broadly accepted best practices in software engineering will be applied to refinement of services in iterative stages, making the software more reliable, interoperable, and maintainable over time. Bamboo's "Shared Services Lifecycle" will describe sequential stages of refinement, and the steps required for services to evolve from one stage to the next. In the most evolved stages of this lifecycle, services will run on geographically distributed instances of a Bamboo Services Platform, which will enable "always-available" guarantees to consumers of those services, and other useful efficiencies.

Partnerships with content providers, and with tool and application developers, will influence characteristics of a Platform. Technology architects will balance characteristics that best support new services and services in the general case, with choices that best align to key standards, practices, tools, and services already adopted by Bamboo's community and other national and international cyberinfrastructure initiatives.

Standardized infrastructure choices and automated procedures for realizing those choices will save money, time, and aggravation for humanists and support staff who must otherwise sink funds and effort into choosing, maintaining, and operating technology, hindering direct engagement with teaching and research.

4.3.2 Description

A Shared Services Lifecycle will establish migration stages through which software will be transformed from narrowly tailored, idiosyncratically accessed, and atypically deployed forms; to a services architecture that conforms to open standards of access, data exchange, and deployment. Application of best software engineering practices, in increasingly rigorous phases, will effect such transformations. Broadly drawn, stages of service evolution can be described as:

  • Early interest in, adoption by, or development by the Bamboo Community
  • Iterative stages of service refinement to addresses concerns like standards-compliance, reliability, and conformity to Bamboo's standard deployment models (some services new to the Bamboo community might already address some or all of these concerns)
  • Deployment on and reliable availability from a set of distributed Bamboo Service Platform instances


A Bamboo Services Platform (Platform) is a technology stack on which services will run in advanced stages of the Shared Services Lifecycle. Such a technology stack will include elements such as a service container; message mediation, transformation, and routing software; a rules engine; connectors to secure, distributed storage; a service orchestration engine; and logging capability to track and report usage metrics. A Platform will be packaged as an "appliance" - a standardized set of technologies bundled together into an easily-replicated server. The Platform appliance will be realized as a virtual machine that is managed via a standard set of interfaces appropriate to distributed ("cloud") hosting. Hosting will be provisioned by commercial, consortial, and/or institutional partners, to best address stewardship, economic, legal, and policy considerations. An appliance may also contain additional technology bundles/systems/platforms (such as Fedora, Nuxeo, JCR interfaces for Plone et al., or SEASR) where the value of co-deployed functionality is judged beneficial in reasonable proportion to operational demands and complexity of the appliance.


In Partnerships with content providers and with tool and application developers, Bamboo will identify software functionality incorporated into existing resources that can be extracted and revised ("refactored") as shared services. Shared services are reusable units of software that deliver functionality needed by multiple projects or tools; and can do so remotely, and independently of the software's implementation language or framework. Bamboo will partner with established digital humanities and related projects to solve reuse and interoperability issues, and to develop a shared services trajectory aligned to core goals and values of the partnerships. Service development and refactoring efforts will prioritize implementation-agnostic interfaces, and will focus on enabling service uptake across initial (disciplinary or other) boundaries of the contributing projects. Partnerships will invest in:

  1. selecting and promoting interoperability standards
  2. identifying and implementing (or refactoring) sharable services from existing software that supports humanist scholarship
  3. refactoring existing projects to use services deployed on the Bamboo Service Platform in lieu of separately developed and maintained functionality
  4. contribution to the Bamboo Atlas of elements related to Partnership services and other Partner-delivered content or functionality, including related scholarly narratives and recipes

Sidebar: Inclusion Challenges

A utopian "Bamboo Services Platform" (Platform) might allow services implemented in any programming language, framework, or paradigm to be deployed and maintained with frictionless ease. A utopian "appliance" might accommodate the inclusion of any tool or system or platform that promises to meet the needs of a broad sector of arts, humanities, and interpretive social science scholarship. Alas, like Galileo's "frictionless plane," such idealized outcomes are unlikely within the real-world constraints of a Bamboo Implementation project. Hard choices will have to be made. It is unlikely that any person or project will be fully satisfied by the complete set of decisions to include in and exclude from a Platform and the appliance on which it is run.

For this reason, it is practical to start with a set of criteria for inclusion and exclusion. These criteria should be designed to scope decisions to deliverables, so that a Platform and appliance address the set of problems that are the focus of a current phase of Bamboo Implementation. As Voltaire wrote, "Le mieux est l'ennemi du bien": the best is the enemy of the good. The Bamboo community will want to inoculate itself against paralyzing overambition.

This proposal does not list criteria, as their establishment is a key element of the work plan for Bamboo Implementation partners; however, the following challenging areas of decision-making relevant to inclusion/exclusion of elements in the Platform and appliance are presented as food for thought. Criteria developed by Bamboo Implementation partners might help to address these challenges effectively, to deliver the greatest achievable value to humanities scholarship:

  • What is the benefit to humanities scholarship of making a system or family of services available as part of a Bamboo appliance, versus making the technology available through some other deployment?
  • What constraints or risks - such as exclusion of valued families of services, or tight-coupling to a single vendor or project team - are associated with a technology?
  • What standards and best practices are implicit (or not) in the adoption of a technology, system, or family of services?
  • How well does a technology stack element align to support requirements of services to be deployed atop the stack in the near future?
  • How does the benefit of including a technology in a Platform or appliance compare to the cost of configuring, operating, and/or managing its inclusion?

4.3.3 Value

Bamboo will deliver technical infrastructure that permits humanities projects to transition from project-specific applications to longer-lived, more broadly supported, more efficiently operated, and more widely used, useful, and interoperable services. This transition will drive toward a future in which content and technology can be can easily discovered, combined, re-mixed, and shared to create new forms of digital research and teaching.

Insert a humanist's perspective on how discovery, combination, re-mix, and shared content/tools will enrich her work

The "cloud" model of service delivery introduces more than an approach to sharing services, connectors, interfaces, and content. Deployment of a Bamboo Services Platform "in the cloud" will minimize risk to individual institutions through guarantees of service availability inherent in well-architected redundancy, drive down data center costs through economies of scale, and leverage pooled software engineering talent.

Insert a CIO's perspective on how cloud models of service delivery are a path to efficiencies in higher ed organizations

Partnering with key content and tool projects that are able and eager to externalize general aspects of their offerings will enable other efforts in digital humanities to decrease the fraction of functionality that must be reinvented. Partners will be credited with contributing to a broader range of scholarship, a value proven by usage metrics collected for services deployed on the Platform. In turn, contributing partners who evolve their offerings to rely on contributed services deployed and maintained in a Bamboo environment will free themselves to focus on areas of specialized value and expertise.

Insert the perspective of a PI on a digital humanities project on how partnership with Bamboo will provide value to each partner

4.3.4 Work Plan

Realizing the Bamboo Services Platform will involve multiple strands of effort that will strongly influence each other in iterative increments. Some effort can (and must) occur in parallel, with appropriate coordination points and processes. Implementation Phase efforts will be informed by Planning Phase Proof of Concept accomplishments and conceptual agreements among Bamboo partners. At a high level, elements of Implementation Phase strands of effort are:

  • Year One
    1. Establish initial qualities and processes for a Shared Services Lifecycle, with attention to proof-of-concept work and preliminary conceptual development among Bamboo Partners
    2. Establish criteria for identifying and evolving partnerships with tool & application projects, and with content providers
    3. Establish criteria for selection of standards, architectures, and technologies to be included in a Bamboo Services Platform (Platform); and for selection of additional technology bundles/systems/platforms to be included in an appliance deployment of a Platform
    4. Establish Year 1 partnerships with tool & application projects, and with content providers, engaged in digital humanities scholarship
    5. Identify initial functionality to be refactored as shared services out of Year 1 partnership project/provider software
    6. Select an initial set (v0.1) of Platform standards, architectures, and technologies, with attention to proof-of-concept work and preliminary conceptual development among Bamboo Partners; focus for the v0.1 Platform will be delivery of services to deliver Bamboo Atlas and Scholarly Networking functionality; alignment with existing cyberinfrastructure initiatives; and, to the extent possible, anticipated platform-support needs for project/provider services.
    7. Explore and address legal and policy issues inherent in distributed hosting of infrastructure and services, in concert with key higher education stakeholders (e.g., Common Solutions Group)
    8. Refactor initial functionality from Year 1 partnership projects/providers into shared services deployable on a v0.2 Platform
    9. Refine criteria for identifying and evolving partnerships with tool & application projects, and with content providers
    10. Realize (implement) the v0.1 Bamboo Service Platform as a cloud-deployable appliance
    11. Deploy pre-production Bamboo Atlas and Scholarly Networking back-end services on the v0.1 Platform
    12. Map v0.2 evolution of the Platform to support services refactored from Year 1 partner projects/providers and further alignment with existing cyberinfrastructure initiatives; also, consider cost/benefit of deploying additional technology on a virtual "appliance" running a Platform
    13. Realize (implement) the v0.2 Bamboo Service Platform as a cloud-deployable appliance
    14. Seek additional partnerships for Year 2
    15. Design, deploy, and prove a service request/redirection mechanism to direct service requests to multiple backing Platforms (or, ideally, clusters of Platforms) from a single point-of-request
    16. Deploy pre-production Bamboo Atlas and Scholarly Networking back-end services, as well as pre-production refactored services developed in partnership with other projects/providers, on the v0.2 Platform
    17. Productionize the latest possible Platform to be released as a cloud-deployed appliance v1.0 by the end of Year 1, with strong prejudice to procure hosting from at least two categories of cloud provisioners (e.g., commercial, consortial, institutional)
    18. Productionize v1.0 release of Atlas, Scholarly Networking, and services refactored from partner projects/providers, with deployment on distributed (cloud-deployed) instances of v1.0 Platforms, where services are accessed through the single point-of-request mechanism
    19. Gather service/platform usage and maintenance metrics and begin to develop a sustainable financial model for running services in the cloud
  • Year Two
    1. Maintain deployed services/platforms/appliances
    2. Further evolve qualities and processes of Bamboo's Shared Services Lifecycle
    3. Establish additional partnerships with tool & application projects, and with content providers, engaged in digital humanities scholarship
    4. Identify functionality to be refactored as shared services out of Year 2 partnership project/provider software, and additional functionality to be refactored out of Year 1 partnership project/provider software
    5. Refactor initial functionality from Year 2 partnership projects/providers into shared services deployable on a v1.1 Platform
    6. Building on Platform v1.0, select v1.1 standards, architectures, and technologies - with focus on evolved Bamboo Atlas and Scholarly Networking service delivery; alignment with existing cyberinfrastructure initiatives; and support of services refactored from Year 1 and Year 2 partner/provider projects
    7. Initiate the full set of work steps described - from definition to production - for an "AlternatePlatform" distinct from the first, as judged a cost-effective investment, to support additional service development platform/framework.
    8. Deploy pre-release evolutions of Bamboo Atlas and Scholarly Networking back-end services, as well as pre-production and pre-release refactored services developed in partnership with other projects/providers, on the v1.1 Platform
    9. Further evolve the Platform (v1.2) toward v2.0 release at end of Year 2
    10. Deploy pre-release evolutions of Bamboo Atlas and Scholarly Networking back-end services, as well as pre-production and pre-release refactored services developed in partnership with other projects/providers, on the v1.2 Platform or v0.2 Alternate Platform
    11. Gather service/platform usage and maintenance metrics and further develop a sustainable financial model for running services in the cloud
    12. Productionize the latest possible Platform to be released as a cloud-deployed appliance v2.0 by the end of Year 2
    13. Productionize the latest possible Alternate Platform to be released as a cloud-deployed appliance v1.0 by the end of Year 2
    14. Productionize v2.0 release of Atlas, Scholarly Networking, and services refactored from partner/provider projects, with deployment on distributed (cloud-deployed) instances of v2.0 Platforms (or v1.0 Alternate Platforms)

Each of the elements listed will have many component pieces of work; these are fleshed out further in the Detailed Work Plans section of this proposal. The high level effort for Year 1 (2010) might be represented graphically, in sequential layers and mutually-influencing strands, as follows.

BIP-Platform-StrandsOfWork-v0

Annual Deliverables

Period

Deliverables

Notes

Year 1
2010

  • Shared Service Lifecycle v1.0
  • Criteria for partnerships and selected partners
  • Criteria for Platform and Appliance technologies, and selection of technologies
  • Platform v1.0, instantiated as an appliance
  • Platform v1.0 Appliance deployed atop two distinct categories cloud-hosting provider infrastructures
  • Productionized community services (Atlas, Scholarly Networking) running on cloud-hosted Platform v1.0
  • Productionized partner-refactored services running on cloud-hosted Platform v1.0
  • Initial draft of business model for sustainable cloud-hosted service delivery, taking account of legal and policy issues
  • Appliance may, if judged beneficial and cost-effective, include technology bundles/systems/platforms in addition to the Platform v1.0

Year 2
2011

  • Shared Service Lifecycle (evolved)
  • Evolved criteria for partnerships and, optionally, additional selected partners
  • Evolved criteria for Platform and Appliance technologies, re-evaluation of initial selections, and evolution of the initial Platform
  • Evaluation of the utility and cost-effectiveness of an alternate Platform, and, if judged a prudent investment, identification of its characteristics.
  • Platform v2.0, instantiated as an appliance
  • Alternate Platform v1.0, instantiated as an appliance (optional)
  • Platform v2.0 Appliance deployed atop two distinct categories cloud-hosting provider infrastructures
  • Alternate Platform v1.0 Appliance deployed atop two distinct categories cloud-hosting provider infrastructures (optional)
  • Productionized, evolved community services (Atlas, Scholarly Networking) running on cloud-hosted Platform v2.0
  • Productionized, evolved partner-refactored services running on cloud-hosted Platform v2.0
  • Productionized partner-refactored services running on cloud-hosted Alternate Platform v1.0 (optional)
  • Actionable business model for sustainable cloud-hosted service delivery, taking account of legal and policy issues
  • An "Alternate Platform," built as a technology stack distinct from the first, will be developed if such an investment is judged cost-effective. The purpose would be to support an additional service development platform/framework substantively different from the first, from which Bamboo's service development community can derive significant value.
  • Should a single, clearly superior cloud-hosting paradigm emerge by Year Two of the Bamboo Implementation Phase, it may no longer be necessary or advantageous to continue exploration of multiple models.



  • No labels