Archive for category Uncategorized

A Radical New Approach to (MUTUAL) Authentication

Posted by Eashwar Ganapathy in Uncategorized on May 19th, 2009

[This thought paper is from Rel-ID Technologies Inc. - a Uniken venture]

Authors Sanjay Deshpande, Dr. Pat Shankar, Eashwar Ganapathy

Abstract In this article, we present a fundamentally new identity framework – RELATIVE IDENTITY - which addresses and eliminates many of the core problems faced by the current identity technologies. We postulate that authentication necessarily has to be mutual and that the only valid way to perform mutual authentication is to make fundamental changes to the identity representation framework.

This can be accomplished by –

  1. Changing from end-point entity labeling (like in the case of login/password, biometric, digital certificates, 2-Way SSL and a combination of these) – to labeling the relationship between the end-point entities (which inherently covers the two end-points in its definition)
  2. Making the authentication protocol truly mutual – and thereby eliminating the susceptibility to man-in-the-middle attacks and phishing

Identity and identification are central to any interaction, both in real and virtual (digital) systems. Especially where the interaction entails access to or manipulation of protected resource(s).

We firmly believe that any identity framework has to address the problem of establishing a mutually-authenticated secure connection BEFORE initiating any data transaction using that connection.
Introduction Identity and Authentication form the central building block of any information security solution/framework. Establishing identity using an authentication protocol is the starting point for any secure transaction. In order to be able to establish identity (be it man or machine), the entity must be characterized by a unique set of symbols (as per the adopted identity representation framework). During the process of actually identifying / authenticating the entity, the same characteristics of  the entity are observed and matched against those that were captured earlier and associated with the entity.

The act of establishing identity is identification. Identity Systems must possess the capability represent, provide, maintain and establish identity. The identity representation framework must ensure that it is extremely difficult to compromise the individual identities it is used to represent. In this article we cover the following points:

  1. Definition of RELative IDentity – the representation
  2. Fundamental properties of identity (representation)
  3. Proof that all authentication must necessarily be mutual ( that 1-way authentication basically flawed)
  4. Fundamental properties of authentication / identification (the process of)
  5. How is Relative Identity different from other identity schemes

The basic flaws and limitations in current identity technologies for websites prevalent in the World Wide Web SSL/Digital Certificates (when used for AUTHENTICATION) become apparent in the context of the axiomatic frame of reference defined in the following sections.

Definition of Relative Identity The relative identity of an entity is

  1. Distributed among the relationship of this entity with other entities. Each such valid relationship –
    • constitutes a unit “Relative  Identity” – an important and inseparable constituent of the identities of each of the entities sharing a valid relationship
    • contributes in the definition of the relative identity of each entity
    • exists only in the context of two (or more) entities who share a relationship
  2. Is the union/collection of all such “Relative Identities”
  3. Is dynamic since new relationships may be established, while old relationships may be discarded, over time
  4. Is associated with a set of labels/attributes/characteristics – immutable and mutable
    • immutable - such as biometrics, which cannot be changed at will
    • mutable - such as SSN which are awarded for a    life time,  log  in passwords, bank account numbers which are changed quite often

In practical implementations of identity based transactions, one is concerned only with the specific (relevant) relative identity and associated attributes, and hence the rest of the identity representation is not susceptible to identity theft.

As is evident from the above definition, the concept of identity in the prevalent conventional identity systems that deal with only labels/attributes/characteristics – “What you have”, “What you know” and “What you are”, totally ignore the most relevant concept of “Who you know” – which is how humans establish trusted relationships.

Fundamental Properties of Relative Identity

The unit relative-identity data -

  1. must be unique (no two relative-identities should have the same identity data)
  2. must be tamper-proof (difficult to reconstruct and reproduce)
  3. must be secret - wholly / partially (should not be communicated in full form during authentication; should be known only to the related entities)
  4. must be used simultaneously and uniquely, to identity all entities involved in the authentication transaction

Most of the prevalent conventional identity systems satisfy properties 1, 2 and 3 above. For example -

  • Login/Password would satisfy 1, 2 (partially) and 3 (partially)
  • Digital Certificates would satisfy 1, 2 and 3 (partially)

What is Mutual Authentication/Identification? Why does one need it?

As yourself the following questions -

  • what is the meaning of authentication if it is not mutual?
  • why would I allow someone to authenticate me, if I can’t authenticate him/her?
  • would I produce my passport to identify myself to someone who does not (even seemingly) possess the requisite authority?
  • Even so, don’t I run the risk of being duped into producing my passport to a person who only looks authentically like he/she has the requisite authority?

The basic flaw in identification over the internet is that an end-user assumes that the website challenging him/her for his/her credentials is indeed the authentic site – so long as interaction with the user-agent application (the web-browser) while accessing the website, is identical to previous such interactions. That is to say – so long as the website looks the same, behaves the same, and does not trigger a negative message  from installed security products (due to more recent efforts in the anti-phishing features of these products).

All things considered, are you sure you can trust such a website that asks for your login credentials?

Conclusion: Authentication, to be of any practical use, MUST BE MUTUAL

Fundamental properties of authentication / identification

The process for identification / authentication

  1. must be tightly integrated with a given/underlying identity representation
  2. must necessarily have a priori access to the identity data that is to be identified / authenticated
  3. must necessarily authenticate all identifying/authenticating parties (entities) – preferably simultaneously

These are simple (minimal) properties that any identity/authentication system must possess. Some of them are straightforward while some may not be seem obvious.

Let us now visit some of the prevalent identification/authentication processes in light of the above properties -

  • Login/Password – satisfies 1 and 2 above
  • Digital Certifications/SSL – does not satisfy 2 and 3, and hence, should NOT be used for authentication
  • Hardware/Software Tokens (and OTPs) – satisfy 1 and 2 but do not satisfy 3

Please note that even the use of multi-factors satisfy only properties 1 and 2 and not the property 3

Let us look at the third property above for authentication protocols that essentially says that - the process MUST be mutual and simultaneous. The term mutual has earlier been defined in the context of client-server architecture as “client must authenticate the  server and  the server must authenticate the client”. Such a definition classifies any “1-way” authentication method executed twice as a valid 2-WAY or mutual authentication process. The fundamental flaws in existing mutual and 1-Way authentication systems are precisely due to the violation of properties (2) and (3) above.
Mutual authentication cannot and should not be implemented using two 1-WAY authentication schemes – e.g. 2-Way SSL, or a combination of login/password and shared secrets/site-key. Any such scheme will be vulnerable to the same attacks that the 1-WAY equivalent is vulnerable to. For example, 2-WAY SSL is susceptible to MITM (man-in-the-middle) in exactly the same way that 1-WAY SSL is - for the same reasons.

How is Relative Identity different from other identity schemes?

Conventionally, identity is associated with the end-point entities (client or server) and authentication involved authenticating the end-points. Authenticating this information for both end-points in sequence is NOT  secure mutual authentication – it is a concatenation of 2 instances of 1-WAY authentication.

The REL-ID (relative identity) approach to authentication is to identify and authenticate the ‘link/relationship’ between the end-point entities – not the individual end-points. That is to say – IDENTITY must necessarily be associated with the ‘link’ representing the relationship between the end-points. This is the only representation, and authentication thereof, that can legitimately be termed as MUTUAL – as the end-points are an integral part of the definition of any such representation.

Authenticating such a ‘link’ would necessarily be mutual – would ensure that all end-points are authenticated simultaneously, and makes the identity of every end-point relative to the other end-point(s) axiomatically.

Conventional Identity System

Conventional Identity System

Relative-Identity System

Relative-Identity System

We believe that, in order to (a) represent the above information correctly at the end-points and (b) arrive at the correct protocols for identification/authentication, one must develop the necessary mathematical frameworks and algorithms. However, before starting to derive them, one must accept and acknowledge the fundamental paradigm shift in the desired properties of such representations and algorithms.

The set of identity representations and identification/authentication algorithms constituting the REL-ID© Security Suite is one such implementation of the identity paradigm described here. Assuming that authentication must necessarily be mutual and simultaneous to be of any value, authentication schemes such as tokens, digital-certificates/SSL, login/password… cannot be compared with REL-ID – since they offer only 1-WAY authentication, at best. Furthermore, methods/products that claim to provide mutual authentication, but in reality implement two 1-WAY authentications (like SITE-KEY – flash-persistent object; Shared Secrets…), will  remain vulnerable to man-in-the-middle attacks due to the inherent vulnerability in the conventional end-point identity representation scheme.

There are no known contemporary technologies/products that are built using mutual authentication protocols, which have the properties mentioned in this article, and which are available commercially.

, ,

No Comments

2-WAY SSL == TWICE PHISHED

Posted by Eashwar Ganapathy in Uncategorized on May 18th, 2009

[This thought paper is from Rel-ID Technologies Inc. - a Uniken venture]

Authors Sanjay Deshpande, Dr. Pat Shankar, Eashwar Ganapathy

Basics of Identity and Authentication - In order to be able to identify / authenticate any entity (be it man or machine), the entity must be characterized by a unique set of symbols, as per the adopted representation scheme. During the process of actually identifying / authenticating the entity, these characteristics are observed and matched against those that were captured earlier and associated with the entity.

The act of establishing identity is identification. An Identity System must be able to represent, provide, maintain and establish identity. The identity representation framework must ensure that it is extremely difficult to compromise the individual identities it deals with. Identity and identification are central to any interaction, both in real and virtual (digital) systems, typically where the interaction entails access to or manipulation of protected resource(s).

For example, we are identified by our name, social security number, passport number, national ID card, fingerprint, voice print, DNA print etc. The context of identification determines the parameters used to determine the identity. While establishing our identity, one or more of these characteristics are elicited / captured from us, and matched against previously captured and stored characteristics.

Let’s take a look at prevalent IDENTITY TECHNOLOGIES

  • Login-Password The login-password is captured and stored A PRIORI with the server and then compared with the login-password that is presented before subsequent interactions with the login-password-secured system.
  • Biometrics (fingerprint/voice/DNA/iris-scan) The biometric is captured and stored A PRIORI and then compared with the biometric data that is presented before subsequent interactions with the biometric-secured system.
  • Photo-ID Cards After verifying that the Photo-ID Card is authentic using a system with a card-reader, the PHOTO on the ID-CARD is matched with the individuals face as well as the system-retrieved photo expected to be on the card.

Note that AUTHENTICATION NEEDS A PRIORI INFORMATION.

PKI, Digital Certificates, SSL and Authentication - PUBLIC KEY CRYPTOGRAPHY, aka Asymmetric Cryptography, is a form of cryptography in which the key used to encrypt a message differes from the key used to decrypt it; the user has a pair of keys - a public key and private key. The private key is kept secret, while the public key may be widely distributed. The keys are related mathematically, but the private key cannot be computationally derived from the public key in ‘reasonable’ time, and vice versa. Messages encrypted with the public key can only be decrypted with the corresponding private key and vice versa. Further, in conjunction with a signing algorithm and a signature-verification algorithm the key pair can be used to send verifiably signed messages

The two main branches of public key cryptography are -

  • Public Key Encryption - A message encrypted with a recipient’s public key cannot be decrypted by anyone but the recipient (using his/her corresponding private key). This ensures confidentiality of messages thus encrypted. An analogy for public-key encryption is that of a locked mail slot. The mail slot is exposed and accessible to the public - its location (the postal address) is, in essence, the public key. Anyone who knows the location can go to the door and drop a message through the slot. However, only the person who possesses the key can open the mailbox and read the messages.
  • Digital Signatures - A signing algorithm, given a message and the private key, produces the signature. And a signature verification algorithm, given a message, its signature and the correct public key, can verify that the message has not been modified with since signing (generation of the signature). This ensures non-repudiation of the message thus sent. An analogy for digital signatures is the sealing of an envelope with a personal wax seal. The recipient checks that the seal is intact and corresponds to that of the sender, before opening the message.

A central problem in public-key cryptography is proving that a public key, which is publicly available, is authentic, and has not been tampered with, or replaced, by a malicious 3rd party. This problem is solved by using a Public Key Infrastructure (PKI), in which one or more 3rd parties, called Certificate Authorities (CA), certify ownership of key pairs. Another approach, used by PGP, is the ‘web of trust‘ method to ensure authenticity of key pairs.

In PKI, a public key certificate (or digital certificate) is an electronic document which incorporates a digital signature to bind together a public key with an identity - information such as the name of a person or organization, their address… The certificate can be used to verify that a public key belongs to an entity. In practice this verification entails verifying that the digital signatures in the certificate were indeed generated using the correct private keys. In a typical PKI scheme, the signature is generated by a Certifying Authority (CA). In a web of trust scheme, the signature is either from the owner (a self-signed certificate) or another user (’endorsements’). In either case, the signatures on a certificate are attestations by the certificate signer that the information in the certificate and the public key belong together.

Why do websites face PHISHING attacks even after adopting DIGITAL CERTIFICATES and SSL technology? What exactly is wrong with DIGITAL CERTIFICATES?

In the 1990’s the DIGITAL CERTIFICATE technology was introduced by VERISIGN bundled with NETSCAPE. The idea was to issue certificates to entities requesting one from VERISIGN. This technology was based on the PKI  scheme (made popular by RSA in the 1980’s). The term Certificate Authority was born and VERISIGN became the first such CA. Eventually other entities could become CAs by purchasing special certificates from VERISIGN or other CA’s and a CA chain came in to existence. The browser technology then invented by NETSCAPE incorporated the certificate technology and along with the SSL protocol became the de facto standard for SECURE INTERNET TRANSACTIONS. Since then this technology, that has been assumed to secure the internet transactions, went on to become a regulatory requirement for most institutions world-over.

Quite frequently, due to evolution, and at times mass acceptance of a technology, the industry seems to overlook some basic but extremely FUNDAMENTAL aspects of technology. Such ignorance (though unintentional) leads to serious security flaws – flaws that are exploited by fraudsters.

DIGITAL CERTIFICATES ARE VERIFIED NOT IDENTIFIED (or AUTHENTICATED) SINCE THERE IS NO DIRECT A PRIORI KNOWLEDGE ABOUT THE CERTIFICATE WITH THE VERIFYING PARTY / PROGRAM.

The Secure Sockets Layer (SSL) and the Transport Layer Security (TLS) protocols use DIGITAL CERTIFICATES to establish a secure connection between a SERVER and a CLIENT. They are  MEMORY-LESS protocols – they were designed to be so in order to make existing and newly developed web applications integrate with them seamlessly - for seamless interoperability. Both protocols are based on CERTIFICATE VERIFICATION. This proves to be a fundamental, subtle and yet non-trivial loophole when used for AUTHENTICATION – there is NO A PRIORI knowledge about the CLIENT or SERVER side certificates available at the verifying side of the connection, during the protocol exchange. A priori knowledge is a fundamental requirement for any AUTHENTICATION PROTOCOL, be it 1-WAY or 2-WAY.

Although the SERVER has a DIGITAL CERTIFICATE which is used to establish a secure SSL connection with the CLIENT – the CLIENT does not have any A PRIORI knowledge of this CERTIFICATE (public key). The SSL protocol only VERIFIES that the CERTIFICATE IS VALID and was issued by the valid CA (as per the contents of the certificate). The equivalent in real life would be to accept an ID card as valid simply because the card has not been tampered with – although the person carrying the card may not be the same person you are trying to authenticate.

Due to this flaw – any application can claim to be the “right or authentic” SERVER to a CLIENT as long as it has a VALID certificate - the same argument can be extended if one is using a CLIENT CERTIFICATE as well (in case of 2-WAY SSL). If 1-WAY SSL protocol is a VERIFICATION protocol – how can 2-WAY SSL protocol claim to eliminate the fundamental issues of AUTHENTICATION – since a 2-WAY SSL PROTOCOL is equivalent to 2 instances of the same 1-WAY SSL VERIFICATION PROTOCOL implemented on both CLIENT and SERVER side.

2-way-ssl-mitm

Have we missed something when it comes to using certificate technology as an identity system for IDENTIFYING WEBSITES?

As per the SSL Protocol, the client confirms that the CERTIFICATE produced by the server is VALID – that the contents of the certificate have not been tampered with, and that the domain name in the certificate indeed is the same as the domain name to which you are currently connected. That is to say, a CERTIFICATE can only be VERIFIED to the extent of the claims made on it – that it belongs to the ENTITY that has presented the CERTIFICATE. However, the client cannot confirm whether it is the SAME ENTITY that the USER is trying to connect to.

A fraudster gets a CERTIFICATE issued to himself/herself, with a domain name that sounds or looks similar, and presents the CERTIFICATE to the user – the SSL/HTTPS layer will NOT be able to tell you whether the USER is  indeed connected to the website he/she wants to connect to. This loophole is not addressed and cannot be addressed in the way the DIGITAL CERTIFICATE TECHNOLOGY and SSL are implemented in the internet today.

Is  it  possible  to  correct  the  present system of DIGITAL CERTIFICATES?

No. Since it would mean a sea change in the entire process of creating, issuing, distributing and identifying the certificates.

If one has implemented the DIGITIAL CERTIFICATE Technology, does that mean their IDENTITY cannot be compromised?

Well, one can make their own educated judgment based on the arguments presented in this article.

According to us, ONCE the IDENTITY has been confirmed, the CERTIFICATE technology could be used to exchange encryption keys, and secure the transaction – IT SHOULD NOT BE USED TO ESTABLISH or AUTHENTICATE THE IDENTITY.

Why not?

The reason why DIGITAL CERTIFICATE TECHNOLOGY is what it is today is because of the fundamental nature of ‘online applications’. Digital Certificates themselves are tools to ensure that a given datum communicated from one end to the other is not tampered with and is ‘signed’ using the private secret corresponding to the publicly available ‘certificate’. The use of digital certificate technology between an all-purpose web-browser and any specific security-critical application can at best be described as a marriage of convenience – essentially because online applications came first!

Conclusions

  1. The fundamental tenet of ‘securing’ any application is to uniquely, unambiguously and reliably identify the user of the application before authorizing and executing any action on the identified user’s behalf.
  2. Furthermore, any centralized and fully automated ‘trust-building mechanism’ for capturing, storing and verifying the trust between ‘essentially anonymous entities’ across wide spectra of businesses and geographies will come with inherent weaknesses – they will be as secure as the weakest link in the security chain built around it.

, ,

No Comments

Website Identity - the root cause for Internet Fraud

Posted by Eashwar Ganapathy in Uncategorized on May 14th, 2009

[This thought paper is from Rel-ID Technologies Inc. - a Uniken venture]

Authors Sanjay Deshpande, Dr. Pat Shankar, Eashwar Ganapathy

Abstract On the internet there are 2 types of websites - ones that take sensitive information from you and ones that don’t. Online banking applications, shopping applications, stock-trading applications are examples of the former; while CNN, Google etc are examples of the latter. This article deals with the very real insecurities of working with applications of the former variety.

How do you know that you are at an authentic website? What if you are not?

Authenticating a website implicitly assumes you already know what to look for in the website, in order to establish the websites identity - which in turn implicitly assumes you know what constitutes the identity of a website. Let us define WEBSITE IDENTITY to be a set of identifiers that can be authenticated to prove the identity of the website. Currently, there are only 3 identifiers that constitute WEBSITE IDENTITY - (1) URL, (2) CONTENT, and (3) SSL/TLS CERTIFICATE. Furthermore, let us note that - any kind of authentication of any identity mandatorily requires a priori knowledge of the identity information.

In this article we shall delve deeper into the above constituents of WEBSITE IDENTITY, and conclusively prove that they are fundamentally incomplete, leaving you - the internet-user, at the mercy of fraudsters who use this knowledge to their benefits, by sending fraudulent emails with links to similar looking websites.

The Website URL - when one sets up a website the first thing one does is to register the domain name (www.mywebsite.com) - that is the primary identifier for the website. More often than not, this is the ONLY identifier - for first-time visitors.

Let us say the website’s domain name is communicated to you by the owner of the website directly/indirectly through trusted channels. In this case, since the identity information for the website has been communicated from a trusted source, the problem of authenticating the website reduces to authenticating the identifier - ensuring that the website you are visiting is authentic.

What if the website’s domain name is communicated to you through channels that you cannot necessarily trust - the first question that arises is how would you know for sure that this identifier indeed belongs to the correct website.

For example, let us assume you are a customer of Bank of America. How would you know for sure that www.bankofamerica.com is the correct website if it has not been communicated to you by the bank? What if someone told you or you searched on the internet and got the website name to be www.boa.com instead, what would you do?

Do you know how to confirm the ownership of a given URL/Domain Name before using it? Further, even if the OWNER (let’s say the bank in this case) told you what to check (assuming there was a way  to get  this  information) - if you have not  received  the information from a TRUSTED source, how do you verify that the information is correct?

NOTE that the DOMAIN REGISTRATION entities (companies that issue the domain name) do not  verify the information provided by the individual who is registering the domain (website address).

NOTE that there is no central TRUSTED repository where you can find the WEBSITE NAME for a given OWNER. There are services like www.who.is – that give you information about who has registered the DOMAIN. However, since the domain registration entities do not authenticate or validate the owner of the website name – which means any one can register any name and own it – what use is this information anyway!

Website Content - Let us move on to the next identifier of the website – the content. Most users confirm the authenticity of  the website based on just the visual cues and verifying the visual cues every time they visit the website.

The very nature of HTTP and HTML (the protocol and language used to retrieve and render website content, respectively) allows one to stream information/content from multiple sources. Which means, even if the visual cues are the same, how do you confirm the validity of the content - what if someone has tampered the content and/or it is being served from somewhere else? Further,  if you are visiting the website for the first time – and the owner has NEVER communicated what the website should look like in the first  place, how would you authenticate the website’s content?

The website content – the HTML document - is available for any one to view, copy and save. It is a trivial task to scrape content off an authentic website (using a browser or wget or website copiers such as HTTrack etc), register a similar (but not identical) website, and put up the same content there -  a COUNTERFEIT WEBSITE.

Let us say there are 2 URLs A and B that are displaying contents Ca and Cb - each implying that the website belongs to the same company X. If the company X has not communicated the correct URL(s) to its users - how would you confirm that the URL you are browsing (A/B) indeed belongs to the company X? If you conclude incorrectly, you could end up providing valuable information to a counterfeit / fraudulent website!

Digital Certificates - By far the most popular method of registering, certifying and verifying identity on internet.

The anti-trust laws in this part of the business world are laughable - to become a valid certification authority one just has to be affiliated to the company called VERISIGN directly or indirectly - isn’t this a fundamental violation of the anti-trust laws? VERISIGN (www.verisign.com) is neither a government nor an international body.

If you navigate to www.patentoffice.nic.in, the official website of India’s Patent Office and visit their e-filing section, most browsers say this could be a fraudulent website - because the certificate is not issued by VERISIGN or any of its affiliated certification authorities (it is issued by a company called NCODE SOLUTIONS - one of the valid root certificate authorities in India). Not just that, the address-bar turns RED.

Who decided that one has to trust VERISIGN? And why does that mean one has to trust all certification authorities certified by VERISIGN? Most consumers do not even know VERISIGN - MicroSoft, maybe, but not VERISIGN. In their defence - given the fundamental flaw with the Certification Authority scheme - there really was no solution but to pick up a bunch of ‘root CAs’ and declare them as the TRUSTED AUTHORITY.

An end-user has no A PRIORI knowledge of the certificate issued to an entity. The transaction (or act) of getting a certificate is a private transaction between the entity (for eg. a bank) and the CA (Certification Authority). This information is never distributed to the end-user and he/she in turn is in no way equipped to verify this information. How can a 3rd party (the CA) vouch for the authenticity of an entity to the end-user? Does it make sense for a child to ask a visitor (a stranger) to vouch for the authenticity of his/her mother? But that is what it is. Why is there a necessity for a 3rd party like VERISIGN (or its derivative CAs) to verify the identity between 2 parties who already share a primary relationship (like bank-customer or enterprise-employee …)?

The original premise of the SSL protocol (that uses these certificates) was to establish an encrypted communication channel between two UNTRUSTED parties. What completely defies logic is - what good would an encrypted channel be between two UNTRUSTED parties?!

Banks and other e-commerce websites boldly declare that they are secure - they have SSL/TLS enabled websites. What good is it when the user does not know what to verify? He/She does not have a-priori knowledge of the information that should be present in the website’s certificate - such as the signed content, the public-key in use etc.

The CA infrastructure and the current SSL protocol violate the basic principles of IDENTITY and AUTHENTICATION - that IDENTITY must be established first and then verified between 2 (now) trusted parties - using information shared during IDENTITY ESTABLISHMENT.

The above discussion conclusively proves that the three primary identifiers – WEBSITE URL, CONTENT and (optionally) PKI-based CERTIFICATES are incomplete as website identifiers. They leave the user at the mercy of fraudsters who use this knowledge to their benefit - sending fraudulent emails with links of similar looking counterfeit websites, luring the ignorant users to divulge sensitive information to the wrong websites.

,

No Comments

Forge.mil: SourceForge-like collaborative development environment

Posted by Shahid N. Shah in Uncategorized on April 25th, 2009

DISA announced earlier this week that Forge.mil is now available to use for unclassified DoD projects. Like Collab.net, SourceForge.net, Google Code, and other collaborative development environments used by thousands of other open source projects, Forge.mil gives you software version control, bug tracking, requirements management and release packaging tools, along with collaboration tools such as wikis, discussion forums and document repositories. If you’re doing community-driven software applications for GOTS, open source, or other non-closed systems you should check out Forge.mil.

No Comments

Effective artifacts to use on modern WOA/SaaS projects

Posted by suvajit_gupta in Uncategorized on November 23rd, 2008

This post summarizes the recommended artifacts in an agile methodology called SPEED (Streamlined Process for Effective Enterprise Development) - see diagram below.

Overview

Let’s start with some overall thoughts before we dive into this busy picture:

  • Grouping: Artifacts are grouped into the functional areas that comprise software project teams: Management, Analysis, Development, Documentation, Testing, and Operations.  Many artifacts related to software products are also created by teams that take the software to market and interface with prospects and customers: Services, Marketing, and Sales.
  • Minimalism: Strictly limit artifacts to only those that directly help conceive and build quality software. Ensure that each artifact has “consumers” who will use the artifact and derive value.  On agile projects, every artifact should be as “lean and mean” as possible.  If a one-page diagram or writeup suffices, that is ideal.  If not, still try to be as brief as possible.
  • Iterations: Most artifacts should be created in an iterative fashion.  They often start out as a simple outline and then get fleshed out as the project proceeds and details become known. Some artifacts (shown in italics) are useful when they are initially created and can subsequently be abandoned as the project proceeds (e.g. a GUI Storyboard doesn’t need to be updated after its purpose is served and the actual GUI is being built).
  • Ownership: While many artifacts are created in a collaborative fashion it is generally recommended that one person “owns” each artifact and acts as its “committer.”  This will ensure the artifact’s conceptual integrity and improve quality.
  • Maintainability: Push technical documentation to lowest level possible (e.g., in code instead of a design document). This vastly improves the chances that such documentation will be read and later maintained as the code matures.
  • Reviews: Only expend limited quality assurance (QA) efforts on artifacts that are project or customer deliverables. For only these artifacts, clarify their compliance with guidelines (use of templates, adherence to standards, etc.)

Note that some artifact names have portions enclosed in angle brackets – this is a notation to indicate a templatized name that should be substituted by an actual name (e.g., “<Component> Design” may be instantiated as “Authentication Design”). Now let’s walk through some high-level commentary about the artifacts shown in the diagram grouped by functional area.

Management

  • As a project is initiated, the Program Manager creates a project Strategy and Initiation Plan to kick things off and gain approval for proceeding.  These two artifacts address broad “why, what, how, and when” questions about the project. The Project Sponsor uses these artifacts to gain approval for the project.
  • The Process Coach creates process artifacts (Methodology, Artifacts diagram, and Internal Release Process) collectively specify the software development lifecycle (SDLC) and periodically refines these as the team “learns” from execution.  At the end of internal releases or the whole project, the team reflects on how things went and captures lessons learned in <Internal Release> Assessments or a Post Mortem Report respectively. These lessons drive ongoing process tuning by the Process Coach.
  • The Program Manager works with the Initiation Team to create a Communications Plan shows how the team members will communicate (meetings/huddles, collaboration tools) and captures how to spread the word about the project both internally and externally.
  • After project approval, the Program Manager works with various Functional Team Leads to create a Release/Staffing Plan. This artifact defines the project’s incremental phases/internal releases and the team structure and staffing/roles. The Program Manager collaborates with the Release Manager to coordinate internal and external releases.
  • As the project kicks off, each Functional Team Lead (for Analysis, Development, Testing, and Documentation) creates a <Functional> Team Plan showing who will do what at a macro level.  For each internal release, Team Plans are subsequently detailed to show tasks/assignments/effort estimates and load balanced on a weekly basis during execution.  Functional Team Leads e-mail a <Functional>Team Weekly Status update to the Program Manager who conducts a weekly team meeting and distributes an overall project Status Report <Week>.

Analysis

  • The Business Architect (typically a domain expert and/or a power user) defines germane terminology in a Glossary, portrays the typical users as Personas, and describes what they want to do with the software via informal User Stories.
  • Product Managers define the project’s high-level Scope and create a compelling presentation showcasing the project’s Highlights.  They should also create a Functional Architecture diagram to illustrate the macro logical components of the system and their interrelationships, while a concomitant spreadsheet explains what each of these components signify. The Analysts also create a Domain Model spreadsheet listing the business entities that the software will operate with.
  • As Business/Functional Analysts work with Architects and Developers to analyze and design the system, they create Concept Models to capture what the user’s mental model is for the software’s underlying concepts.  The Wireframe Modeler creates wireframe GUI layouts packaged into a GUI Storyboard, while styles and branding are specified via a Visual Design Concept by the Graphics Designer.  Detailed user interactions are modeled via <Component> GUI Statecharts and other specifications are captured as <Component> Requirements – these artifacts are started by Analysts but completed by Architects and Developers during development.

Development

  • Early on in the project, Architects evaluate macro technical choices via <Alternatives> Evaluation matrices (scores against weighted criteria).  Developers may be involved to build throwaway Prototypes to explore details of some of these technical alternatives.
  • Before much code is written, the Build Engineer should work with Developers to define and automate Configuration Management (CM) and Continuous Integration (CI) and capture relevant tools/steps in a Build Process document.
  • The high-level structure of the software should be captured by Architects in a Technical Architecture diagram & document.  Architects and Developers then create <Component> Design documents to propose, solidify, and explain the non-trivial aspects of the code.  How existing code or customers will be affected may be formally analyzed and documented in Impact Analysis writeups jointly created by Architects and Developers.  If security is important to the project, a formal architectural risk analysis by a Security Architect will result in a Threat Model outlining the key vulnerabilities/Abuse Cases of the system, which then should be mitigated via design or code changes.  To help explain the system <Component> Object/Data Models are often created by Architects and Developers, initially “as planned” and finally “as built.”
  • Various technical standards (API Strategy, <GUI, Server, Database> Coding Standards, and GUI Style Guidelines) should be defined by Architects and Developers early on in the project and refined as the design and code evolves.
  • If there is legacy or pre-existing code that will be leveraged by the project, a Code Reuse Assessment writeup capturing components, their responsibilities, and key collaborations is very helpful.  Data conversion steps for upgrading customers are captured in a Data Conversion Plan.
  • An <Internal Release> consists of demonstrable code and a growing set of unit tests will eventually grow into the External Release at the end of the project.  <Internal Release> Notes at the end of each cycle are used to communicate what features, fixes, and known limitations people should be aware of as the growing codebase is tested.
  • As external components are incorporated into the code, Developers must add them to the Off-the-Shelf Components list to ensure proper licensing and version tracking/upgrades.

Testing

  • The Testing Team Lead creates a risk-driven Test Strategy document to specify the overall approach and needs for testing including environment/tools, automation strategy, and quality metrics.
  • The Testing Team Lead creates an Issue Tracking Workflow & Submission Checklist to clarify how issues (defects and/or enhancements) are submitted, fixed, verified and to specify severity level definitions and need for supporting information when submitting issues.
  • The Testing Team Lead and Testers collaborate to create <Area> Test Plans for various functionality and non-functional areas which are subsequently fleshed out by Manual Testers as Test Cases/Data or by Test Automation Specialits as Regression Test Scripts/Data.
  • The Testing Team Lead delivers internal testing results via a Test Execution Report detailing what tests passed or failed and comments from Testers and output from failed test scripts.  If external validation for specialized areas is sought, they result in <Usability, Security, Performance> Assessment Reports.

Documentation

  • Technical Writers produce User Documentation such as online help/tutorials, user/training guides, and external release notes.
  • Many systems also ship with Product Content like tips, best practices, templates, and samples which are typically created by Professional Services staff.

Operations

  • The Operations Manager creates a Deployment Strategy to define how the software will be moved into production and to map out the various configurations needed for the live system.
  • The Operations Manager should also create an Operational Readiness Checklist to ensure that all the i’s are dotted and t’s are crossed as the system is moved into production.

Services

  • Most enterprise-class software requires a set of Professional Services Offerings defined by the Professional Services Team who also is responsible for creating specific Customer Upgrade Strategy documents and Implementation Plans.

Marketing

  • The Marketing Manager creates a Product Launch Strategy to plan the activities surrounding the “go to market” timeframe.
  • The Marketing Specialist creates Product Collateral to assist in sales efforts.

Sales

  • The Sales Manager in collaboration with the Sales Account Executives creates a Product Sales Strategy to clarify the target markets and pricing.
  • The Sales Support Specialist creates a Sales Presentation and Sales Demo as the project is wrapping up so that these will be ready alongside the finished software.

Artifacts and their interrelationships

No Comments

The State of the Scripting Universe

Posted by Shahid N. Shah in Uncategorized on September 5th, 2008

As architects we’re routinely asked about the “best tool for the job.” More and more these days the answer is a scripting language like Perl or Ruby — and, without a wink or cringing about it. Even as little as ten years ago those of us who liked scripting languages only recommended it for small utilities or daily script work — however, more and more these days scripting languages are being used to deliver entire services in a SOA or WOA environment and even build complete applications. I still think that scripting is for specialized domain-specific uses but others believe that scripting can be used for general-purpose use cases, too.

Check out this CIO Magazine article “PHP, JavaScript, Ruby, Perl, Python, and Tcl Today: The State of the Scripting Universe“. It’s got some good content and thoughts from the creators or managers of the projects that the scripting languages come from.

No Comments