Singleton Pattern

Singleton is one of the most basic and easy to use design pattern from the Gang of Four book of design patterns. The GOF describe the singleton pattern as "Ensure a class has only one instance, and provide a global point of access to it". Singleton pattern requires to ensure that only one instance of a class is created in the Java Virtual Machine.

public class Singleton {
 
    // Static member holds only one instance of the Singleton class
    private static Singleton instance;
 
    // Singleton prevents any other class from instantiating
    private Singleton() {
    }
 
    // Providing Global point of access
    public static Singleton getInstance() {
        if (null == instance) {
            instance = new Singleton();
        }
        return instance;
    }
}

Singleton pattern is generally implemented using static factory method called getInstance, static variable holding a class instance and a private constructor. This ensures that singleton instance is created only when required also known as lazy instantiation. The private constructor inhibits sub-classing of the singleton class which if intentional should be made final class or added into an isolated singleton package.

Mutliple classloaders still will be able to have multiple singleton instances as the classes are loaded by different classloaders access a singleton. Hence in multiple JVM environment, each of them will have their own copy of the singleton object which can lead to multiple issues specially in a clustered environment where the access to the resource needs to be restricted and synchronized. The basic method to handle such case is to load the classloader which in turn loads the singleton instance as below.

private static Class getClass(String classname) throws ClassNotFoundException {
      ClassLoader classLoader = Thread.currentThread().getContextClassLoader();
      if(classLoader == null)
         classLoader = Singleton.class.getClassLoader();
      return (classLoader.loadClass(classname));
   }
}

There are multiple other techniques such as JMS, DB, Custom API and 3rd party tools which handle such scenario but they also impact the business logic. Tools like Terracotta and Oracle Coherence work on the concept of providing an in memory replication of objects across JVMs in effect providing a singleton view or making use of any of the cluster-aware cache provider’s like SwarmCache or JBoss TreeCache. Application servers provide some level of custom API’s to circumvent the problem such as JBoss HASingleton Service, Weblogic's concept of Singleton Service and WebSphere's singleton across cluster in the WebSphere XD version.

Also if the singleton implements java.io.Serializable and serialized the object then the subsequent deserialize calls over multiple times would end up with multiple instances of singleton class. The best way to handle this scenario is to make the singleton as an enum enabling the underlying java implementation to handle the details. Other work around would be to implement the readResolve method (or readobject() method) in Singleton class to return the same singleton instance. The clone method is overridden to throw an exception preventing multiple instances of singleton class using cloning.

Also the getInstance method is not thread safe since multiple threads can call getInstance method in parallel creating multiple class instances. The multithreading access can be handled by declaring the getInstance method as synchronized. But synchronized method in turn adds an extra overhead for every call to the getInstance method since synchronized methods can run up to 100 times slower than unsynchronized methods. This can be avoided using the double checked locking as below which checks for synchronization for the very first call to getInstance method when the class instance is not initialized. The double null check is required in the case were a thread can be preempted after entering the synchronized block, before it can assign the singleton member variable, with the subsequent waiting thread to enter the synchronized block. Also when multiple threads tries to read the singleton instance, there is a possibility that a stale or partially initialized value is read by a thread, causing it to create a new instance. The volatile variable from java 5 guarantees that all the writes will happen on volatile variable before any reads, hence avoiding creation of another instance due to race condition.

Unfortunately the double-checked locking is not guaranteed to work because the compiler is free to assign a value to the singleton member variable before the singleton's constructor is called.

public class Singleton {
 
    // Singleton instance should we volatile to avoid reading stale value
    private static volatile Singleton instance;
 
    // Singleton prevents any other class from instantiating
    private Singleton() {
    }

    public static Singleton getInstance() {
       if (null == instance) {
          synchronized (Singleton.class){
              if (null == instance) {
                instance = new Singleton();
              }
          }
       }
       return instance;
    }
}

To provide a fast, easy and thread safe singleton solution, we initialize the class instance by static variables which guaranteed that they are executed only once during the first time when they are accessed.  The Classloader guarantees that singleton instance will not be visible until its fully created. The below implementation however compromises the fact that singleton instance can't be changed later to allow multiple instances of Singleton class. The below implementation specifies singleton during compile time as opposed to runtime, a singleton registry implementation can be used which maintains the static HashMap containing the class name as key and the class instance (instantiated using reflection) as the value.

// Singleton Class
public class Singleton {

 // declare a static instance of Singleton class to initialize only once during loading. 
    private static final Singleton instance = new Singleton();
  
    // declare the constructor as private
    private Singleton() {}
 
    // get the instance of singleton class
    public static Singleton getInstance() {
        return instance;
    }
}

Another approach to implement singleton which is considered much easier is using enums. Enum guarantees thread-safety by default hence there is no need for double checked locking during instance creation of singleton class. Another problem with conventional implementation of singleton is that once we implement Serializable interface they no longer remain Singleton because readObject() method always returns a new instance just like constructor. This can be avoided by overriding readResolve() method to return the singleton instance and discarding the newly created instance. This can become even more complex if the Singleton class maintained state, were the instance becomes transient. With the Enum approach, as the since enum instances are by default final, it provides safety against multiple instances due to serialization.

// Implement a singleton using enum which is accessed as Singleton.INSTANCE
public enum Singleton{
    INSTANCE;
}

Some of the criticism towards singleton is that they increase the dependencies by adding global code. They make the code tightly coupled, violate single responsibility principle and also maintain their state for the lifetime of the application which complicates unit testing (by initializing singletons in specific order for unit testing). On the other hand singletons are used extensively to ensure that there are no duplicate instances of the class thus avoiding OutOfMemory issues. Hence the default scope for spring beans is singleton since any stateless object may be singleton without causing any concurrency issues.
The most commonly accepted usage of Singletons were they yield the best results is in a situation where various parts of an application concurrently try to access a shared resource such as a Logger.

CXF WS Security

1. Setup the WS Security in Weblogic
2. Test it using SOAP UI Client
3. Create CXF Client to Send Request with BST
4. Receive the Response from CXF Client with Security Confirmation

Setting up WS Security in Weblogic

Oracle Weblogic Server 12c was used to configure with the client application. The client application is the EJB application with an EJB Stateless bean. It uses weblogic.jws.Policies and weblogic.jws.Policy classes to specify the location to the policy file.

@Stateless
@WebService(targetNamespace="http://ws-connector.sp.ttp.tsm.com", name="TTPSPService",
  portName="TTPSPPort", endpointInterface="com.tsm.ttp.sp.ws_connector.TTPSP")
@Policies( { @Policy(uri = "policy:TTPSP-Policy.xml") } )
public class TTPSP  implements com.tsm.ttp.sp.ws_connector.TTPSP {

    public TTPSP() {     }

   @Override
   public CheckEligibilityResponseType checkEligibility(CheckEligibilityRequestType input) {
 CheckEligibilityResponseType result= new CheckEligibilityResponseType();
 return result;
   }
   ...........
}

The policy file located in the “Project/ejbModule/META-INF/policies” folder. The policy.xml specified that it requires “WssX509V3Token11” to the Recipient which is believed to be the Binary Security Token. The algorithm-suite preferred was “Basic256”. Timestamp must be included and, both the Headers and Body be signed entirely. Also specified the requirement of security confirmation using “<sp:RequireSignatureConfirmation/>” element.

To generate the build first we need to generate the “wlfullclient.jar” for the current weblogic server. The JarBuilder is used to create wlfullclient.jar using the following command.
WL_HOME/server/lib> java -jar wljarbuilder.jar

In some cases the "weblogic.jws.Policies" or other packages maybe absent in the wlfullclient.jar recently created. In case of Weblogic 10.3.3, the packages weblogic.jws.Policies and weblogic.jws.Policy are not present in the “wseeclient.jar” and “wls-api.jar” jar files either. These packages (weblogic.jws.Policies) can be found in the jar file

“C:\oracle\Middleware\modules\*ws.api_1.0.0.0.jar*” for Weblogic 10.3.3 and in

“C:\Oracle\Middleware\modules\ws.api_2.0.0.0.jar” for Weblogic 12.1

In Eclipse, we create a “New EJB Project” and all the source packages are added in the “ejbModule” folder. Also the “ejbModule” contains the “META-INF” folder containing the “MANIFEST.MF” and the “policy.xml” files. Create a “lib” folder in the project and add the “wlfullclient-11.1.jar”, “ws.api_1.1.0.0.jar” and other jars required.

Now create a “New Enterprise Application Project” i.e. EAR Project naming it same as previous project with EAR appended at the end. During the creation configure the EAR settings to add J2EE module dependencies. The EAR Project can be created by right clicking the “Deployment Descriptor: Projectname” -> New -> Project -> EAR Project.

In order to configure the weblogic server with WS Security, we need to generate a keystore using java keytool as follows:

1)  Generate a new JKS Keystore with new Keypair:

keytool -genkeypair -alias bank: BANK -keyalg RSA -keysize 1024

        -validity 365 -keystore bank.jks

KeyStore Password: t1bank

                Enter key password for <bank: BANK>: t1bank

2)  Export a certificate from the generate keystore:

keytool -exportcert -alias bank: BANK -file bank.cer -keystore bank.jks

Enter keystore password:

Certificate stored in file <bank.cer>

Now to configure the keystore in weblogic we have two choices, one is to add the keys from the bank.jks keystore to the DemoTrust.jks keystore. The other option is to change the Keystore configuration to use the Custom keystore.

   Initially, we tried to setup a custom keystore using the description from this link. The process was as follows:

1. In Weblogic server administration, expand Servers and select the server you need to update.
2. Select Configuration -> Keystores -> SSL.
3. Click the Change link under Keystore Configuration.
4. Select Custom Identity and Java Standard Trust as the keystore configuration type and continue.
5. For the Custom Identity Keystore File Name, enter the path to your Java keystore. Select Keystore type as jks .
6. Enter your Custom Identity Keystore Passphrase as the password you used when you created the Java keystore
7. Confirm the password, click Continue and then Finish. 
8. Go back under Servers and select the server that you are working with.
9. Select Configuration -> Keystores -> SSL.
10. Under Configure SSL, select Keystores as the method for storing identities.
11. Enter the server certificate key alias (in this example, myalias was used), and the keystore password
12. Click Finish to finalize the changes. You will need to reboot Weblogic for those changes to take effect.

After going with the above approach by changing the "Keystore Configuration" to "Custom Identity and Java Standard Trust" and setting all the JKS Keystores pointing to bank.jks, weblogic console gave the following error:

"weblogic.management.DeploymentException: Deployment could not be created. Deployment creator is null."

The reason for the above error turns to be that the SSL configuration is not been updated corresponding to the Keystore configuration. Hence the “SSL Configuration" was configured to use a “Custom Trust Store” and the Key Alias and Password to be used were specified. It resulted in a failure, as no request was able to hit the Weblogic server.

After learning from the above failures, we swap to the first option, i.e. add the certificate to the DemoTrust.jks. The Demo keystores are the Keystores configured in Weblogic Console by Default. The Demo keystores are configured under (Environment-> Servers-> AdminServer-> Configuration-> Keystores). The names of the Demo keystores and their default passwords are as follows:

Keystore: DemoTrust.jks

Password: DemoTrustKeyStorePassPhrase

Path:     C:\Oracle\Middleware\wlserver_10.3\server\lib

Keystore: DemoIdentity.jks

Password: DemoIdentityKeyStorePassPhrase

Path:     C:\Oracle\Middleware\wlserver_10.3\server\lib

Keystore: cacerts

Password: changeit

Path:     C:\Oracle\Middleware\jdk160_21\jre\lib\security

 All the Demo keystores for the Weblogic server are located in the path “Oracle\Middleware\wlserver_10.3\server\lib”. One could find the "DemoTrust.jks" and "DemoIdentity.jks" files here. Here we add the bank.cer ONLY TO “DemoTrust.jks” keystore and NOT TO “DemoIdentity.jks”. We DON'T ADD bank.cer to "cacerts" in located in “Oracle\Middleware\jdk160_21\jre\lib\security” folder too. The process is as follows:

1)     Add the bank.cer ONLY TO DemoTrust.jks keystore using the following command:

      keytool -importcert -alias bank: BANK -file bank.cer -keystore DemoTrust.jks

      Enter keystore password: DemoTrustKeyStorePassPhrase

        Trust this certificate? [no]: yes

        Certificate was added to keystore

2)    We confirm if the keys are added into the DemoTrust.jks by the following command:

        keytool -list -keystore DemoTrust.jks 

All the server logs can be found in the following log file:

“Oracle\Middleware\user_projects\domains\base_domain\servers\AdminServer\logs\base_domain.log”.

WS Security with BST Client using SOAP-UI

Open the SOAP-UI and create a new project based on the WSDL or Endpoint provided.  In order to set WS Security for the SOAP-UI client, right click on the project created and select “Show Project View” from the Menu.

Select the “WS-Security Configurations” tab and select the “Keystores/Certificates” tab in the inner window.  Then click on the “+” button to select the new Keystore and enter the Keystore password.

Then Select the “Outgoing WS-Security Configurations” tab in the inner window.  Click the add button from the top section to add a new Configuration in the outgoing WS-Security configurations.  Fill in the Default Username/Alias and password to be used in all the WSS Actions.  Now in the bottom section click the “+” button to add a Timestamp Entry. Fill the “Time to Live” as 1800000 and check the option to set the Millisecond Precision of the Timestamp.
      Moving forward, add the second WSS Entry “Signature” which will be creating the Binary Security Token. Select the Keystore which is been entered in the “Keystores/Certificates” section and enter the Alias name with the corresponding password. Select the Key Identifier Type as “Binary Security Token” in order to create the Binary Security Token  first. Select the signature algorithm, canonicalization algorithm and the digest algorithm.  At last check “Use single certificate for signing” in order to use only the base certificate and not all the certificates in the chain. The “Parts” section is kept empty, but by default SOAP-UI will sign the “Body” element using the generated BinarySecurityToken.

Moving forward, add another Signature WSS entry, the third one of all. Similar to the previous Signature configuration, select the keystore, enter alias and password, and select the same entries for the algorithms as before. Most importantly for the Key Identifier Type select “Issuer Name and Serial Number” in order to sign all the elements. The “Use single certificate for signing” option remains unchecked as all the certificates in the chain should be used for signing.  Unlike before, use the “+” button near the “Parts” section to add the “Timestamp”, “Body” and “BinarySecurityToken” elements with the namespace and encoding information (Default its “Content”).

Now the project is WS Security enabled for the Requests. But before firing the individual requests, select the Request Method and under project properties make sure that “Strip whitespaces” Property is set to “true”.

Then click the “Authentication and Security-related settings” for the Request at the bottom causing a window being opened. Select the Outgoing WSS as the same name given in the “Outgoing Security Configurations” section before.

This can also be done by right clicking the request and using the menu to select the “Outgoing WSS” to the corresponding Outgoing WS Security configuration. Mostly the prior method is preferred.

The Resulting SOAP-UI Request is as follows:

   
     
       
         
         
         
         
           
             
           
           
           0up9O5yZ6wLnau/eTzPZtfz+IIM=
         
         
           
             
           
           
           EAuvZTemCXTia8fPYXngIZOCPE0=
         
         
           
             
           
           
           kYMlR5YhU9CHpVaL0uCVnxINNF0=
         
         
         FSSax.....CWtoxx0=
         
        
         
           
              CN=BANK,OU=BANK,O=BANK,L=SG,ST=SG,C=SG 
              1329894156
           
       
     
     
   
   
           l4TLCUURhrJbRjXEIEGirTpg==
   
   
     
       
       
       
         
           
      
         
         EAuvZTemCXTia8fPYXngIZOCPE0=
       
     
     TmjGBLZJ69kHZNG8=
     
        
          
        
    


    2012-02-22T08:07:59.352Z
    2012-03-14T04:07:59.352Z
 
 
 
 
      
            1222
            232
            CheckEligibility
      
 

From the above request format received from SOAP-UI for the WS Security enabled Server, we could point out some of the key things. First, the Security Header inside the Soap Header contains the following elements:

1. Signature 1
2. BinarySecurityToken
3. Signature 2
4. Timestamp

(highlighted in Blue above) while Signature 2 consists of <ds:KeyInfo> Element.

       In Signature 1 we find the <ds:X509Data> Element (highlighted in Blue) inside the <wsse:SecurityTokenReference> element. The <ds:X509Data> Element contains the <ds:X509IssuerSerial> element.  From its name it suggests that this is signed by the IssuerSerial KeyIdentifier. Also in Singature 1 element we find three <ds:Reference> elements assumed to be signatures (from the order of Signature Parts specified in SOAP-UI) as follows:

1. TIMESTAMP                       : <ds:Reference URI="#Timestamp-8">
2. BODY                                   : <ds:Reference URI="#id-10">
3. BINARYSECURITYTOKEN:  <ds:Reference URI="#CertId-2B6B2C4066C46E9954132989807937513">

    In Signature 2 on the other hand we see just the <wsse:Reference> element inside the <wsse:SecurityTokenReference> element. The  <wsse:Reference> element has the ValueType as “X509v3” which suggests that this is signed by the BinarySecurityToken. Even though we didn’t specify any values for “Parts” section in the first Signature using BinarySecurityToken as KeyIdentifier, we see one <ds:Reference URI="#id-10"> element assumed to be a signature.  Comparing the URI of the Reference element with Signature 1 element signatures, we assume that it is the Signature of the Body Element. Hence even if the Signature Parts is empty, by default the Body element is signed by Default using the specified KeyIdentifier.

Create CXF Client to Send Request with BST

    One of a Senior developer Xei Songwen provided an implementation of WS Security using which just signed Body element to send the request. The classes contained a Dispatcher, Client, Customized WSS4JOutInterceptor implementation, PasswordCallback, SigningCheck.properties and the Spring configuration described in the Class Diagram.

The following Jar Issues were faced and resolved while testing the application initially:

ERROR:
Caused by: java.lang.NoClassDefFoundError: org.apache.axiom.soap.impl.dom.soap11.SOAP11Factory
       at org.apache.axis2.saaj.SOAPPartImpl.<init>(SOAPPartImpl.java:209)
       at org.apache.axis2.saaj.SOAPPartImpl.<init>(SOAPPartImpl.java:246)

ADDED:  saaj-impl-1.3.2.jar
REMOVED: axis2-saaj-1.4.jar

ERROR:

Caused by: java.lang.NoClassDefFoundError: com.sun.org.apache.xerces.internal.dom.DocumentImpl

       at java.lang.ClassLoader.defineClassImpl(Native Method)

       at java.lang.ClassLoader.defineClass(ClassLoader.java:223)

ADDED:  xercesImpl-sun-version.jar

ERROR:

Caused by: java.lang.IncompatibleClassChangeError

       at org.apache.xalan.transformer.TransformerIdentityImpl.createResultContentHandler(TransformerIdentityImpl.java:207)

       at org.apache.xalan.transformer.TransformerIdentityImpl.transform(TransformerIdentityImpl.java:330)

       at com.sun.xml.messaging.saaj.util.transform.EfficientStreamingTransformer.transform(EfficientStreamingTransformer.java:423)

       at com.sun.xml.messaging.saaj.soap.EnvelopeFactory.createEnvelope(EnvelopeFactory.java:136)

       at com.sun.xml.messaging.saaj.soap.ver1_1.SOAPPart1_1Impl.createEnvelopeFromSource(SOAPPart1_1Impl.java:102)

       at com.sun.xml.messaging.saaj.soap.SOAPPartImpl.getEnvelope(SOAPPartImpl.java:156)

       at com.sun.xml.messaging.saaj.soap.MessageImpl.getSOAPBody(MessageImpl.java:1287)

       at

ADDED:  saaj-api-1.3.2.jar

The spring configuration for the WSS4JOutInterceptor is as follows:

When tried to use the KeyIdentifier as “DirectReference” or “IssuerSerial” in the Single WSS4JOutInterceptor and specified BinarySecurityToken element in the “signatureparts” as specified above, it gave the following error:

Caused by: org.apache.ws.security.WSSecurityException: General security error (WSEncryptBody/WSSignEnvelope: Element to encrypt/sign not found: http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd, BinarySecurityToken)
at org.apache.ws.security.message.WSSecSignature.addReferencesToSign(WSSecSignature.java:588)
 at org.apache.ws.security.message.WSSecSignature.build(WSSecSignature.java:769)
 at org.apache.ws.security.action.SignatureAction.execute(SignatureAction.java:57)

In order to tackle the problem of missing BinarySecurityToken element in the SecurityHeader before the Interceptor tries to sign the BST (BinarySecurityToken) element,  BST element is added before the request is passed to the inoke() method. The code added is as follows:

final String XMLNS_WSU = "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd";
final String XSD_WSSE = "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd";

final SOAPFactory sf = SOAPFactory.newInstance();
final SOAPElement securityElement = sf.createElement("Security", "wsse", XSD_WSSE);
final SOAPElement authElement = sf.createElement("BinarySecurityToken", "wsse", XSD_WSSE);
authElement.setAttribute("EncodingType", " http://.....1.0#Base64Binary");
authElement.setAttribute("ValueType", "http://.....1.0#X509v3");
authElement.setAttribute("wsu:Id", "CertId-CA440EE13ADE87BAE5133044746778913");
authElement.addAttribute(new QName("xmlns:wsu"), XMLNS_WSU);
authElement.addTextNode("SMDFhdffIUSDFJL9090ddf213asdsKFHkfdfgjfs234gbhfg56icxdd24rgd"));
securityElement.addChildElement(authElement);
soapRequest.getSOAPHeader().addChildElement(securityHeader);

But instead of detecting the BST element and trying to sign it, the WSS4JOutInterceptor throws the following exception:

org.apache.xmlbeans.XmlException: error: Attribute "Id" bound to namespace "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" was already specified for element "wsse:BinarySecurityToken".

Considering the suggestion given from the CXF Forum two interceptors extending WSS4JOutInterceptor were configured. The first one is configured with the KeyIdentifier as “DirectReference”, while the second one configured as “IssuerSerial”. Now the BinartSecurityToken (BST) was generated but nothing was signed. Also the Issuer Serial along with Timestamp elements was absent in the SecurityHeader. On reversing the KeyIdentifier values across the two interceptors, BinarySecurityToken vanished but all the previously missing elements reappeared.  This lead to a suspicion that only the first interceptor was being called while the second interceptor remained unexecuted.
      After running the debugger numerous times, the doubt was confirmed. As mentioned by somebody in the forum that the instance names of the two interceptors along with their class names should be different in order for them to be executed. But still success remained far off. One doubt still pondered that both the interceptors extend the WSS4JOutInterceptor for all their functionality with just the Class name different.
    After looking at the source code of org.apache.cxf.ws.security.wss4j.WSS4JOutInterceptor below, it seems to be a possibility that getId() method of the WSS4JOutInterceptorInternal Class is called before calling the handleMessage() method of the inner class. This handleMessage() method (line 257) in turn calls the doSenderAction() method defined in the org.apache.ws.security.handler.WSHandler Class.

public class  WSS4JOutInterceptor extends AbstractWSS4JInterceptor {
       ...................................

   private WSS4JOutInterceptorInternal ending;

   public  WSS4JOutInterceptor() {
         super();
         setPhase(Phase.PRE_PROTOCOL);
         getAfter().add(SAAJOutInterceptor.class.getName());
         ending = createEndingInterceptor();
   }

       ...................................
   final class  WSS4JOutInterceptorInternal implements PhaseInterceptor {
            ...................................
     public void  handleMessage(SoapMessage mc) throws Fault { …………. 
           doSenderAction(doAction, doc, reqData, actions, somebooleanvalue);
     }
            ...................................
     public String  getId() {

            return WSS4JOutInterceptorInternal.class.getName();
     }
            ...................................
   }
}

If the getId() method of the WSS4JOutInterceptorInternal Class is altered to return a different class name rather than the actual one, then following exception is thrown:

SystemErr     R javax.xml.ws.soap.SOAPFaultException: Unknown exception, internal system processing error.
SystemErr     R      at org.apache.cxf.jaxws.DispatchImpl.mapException(DispatchImpl.java:235)
SystemErr     R      at org.apache.cxf.jaxws.DispatchImpl.invoke(DispatchImpl.java:264)
SystemErr     R      at org.apache.cxf.jaxws.DispatchImpl.invoke(DispatchImpl.java:195)

When a new Interceptor (MSMBSTWSS4JOutInterceptor) imitating the same code copied from WSS4JOutInterceptor is added along with the old Interceptor (MSMWSS4JOutInterceptor) extending WSS4JOutInterceptor, then both the Interceptors are invoked one after the another. Hence the first Interceptor creates the BinarySecurityToken while the second Interceptor (extending WSS4JOutInterceptor) signs all the elements including the BinarySecurityToken created before.

The same issue of signing the BinarySecurityToken can be resolved by overriding the Apache CXF and SAAJ classes. We first override the doSenderAction() method of the WSHandler Class in the BSTWSS4JOutInterceptor implementation.

public class BSTWSS4JOutInterceptor extends WSS4JOutInterceptor {

 private static final String msmActionClass = “org.example.BSTSignatureAction”; 

 protected void doSenderAction(int doAction, Document doc, RequestData reqData,
                               Vector actions, boolean isRequest){

     boolean mu = decodeMustUnderstand(reqData);
     ............
     ............
     for (int i = 0; i < actions.size(); i++) {
       int actionToDo = ((Integer) actions.get(i)).intValue();
       ............
       switch (actionToDo) {
       case WSConstants.UT:
       case WSConstants.ENCR:
       case WSConstants.SIGN:
       case WSConstants.ST_SIGNED:
       case WSConstants.ST_UNSIGNED:
       case WSConstants.TS:
       case WSConstants.UT_SIGN:

    if (isBSTEnabled && actionToDo == WSConstants.SIGN) {
  
           Action doit = null;
           
           try {
            doit = (Action) Loader.loadClass(msmActionClass).newInstance();
           } catch (Throwable t) {
               if (log.isDebugEnabled()) {
                 log.debug(t.getMessage(), t);
               }
               throw new WSSecurityException(WSSecurityException.FAILURE,
               "unableToLoadClass", new Object[] { msmActionClass }, t);
           }

    if(doit != null) {
  doit.execute(this, actionToDo, doc, reqData);
    }

  } else {
  wssConfig.getAction(actionToDo).execute(this, actionToDo, doc, reqData);
   }

         break;

       case WSConstants.NO_SERIALIZE:
                reqData.setNoSerialization(true);
                break;
       default:
                Action doit = null;     
     ............
     ............
 }
}

Now in the overridden BSTSignatureAction class we override the implementation of the execute() method inorder to change the WSSecSignature class to BSTWSSecSignature class as follows:

public class BSTSignatureAction implements Action {

 public void execute(WSHandler handler, int actionToDo, Document doc,
                     RequestData reqData){

     String password = handler.getPassword(...).getPassword();
     BSTWSSecSignature wsSign = new BSTWSSecSignature();
     ............
     ............
     ............
     try {
      wsSign.build(doc, reqData.getSigCrypto(), reqData.getSecHeader());
      reqData.getSignatureValues().add(wsSign.getSignatureValue());
     }
     catch() { ... }
 }
}

At last we override the Now in the overridden BSTSignatureAction class we override the implementation of the execute() method inorder to change the WSSecSignature class to BSTWSSecSignature class as follows:

public class BSTWSSecSignature extends WSSecBase {

............
 public Document build(Document doc, Crypto cr, WSSecHeader secHeader)
                 throws WSSecurityException{

     ............ 
     // call addBST() method, a duplicate of prepare() method were keyIdentifierType is 
     // considered only as BST_DIRECT_REFERENCE in its switch case.
     addBST(doc, cr, secHeader);
        
     // create an empty vector for signature parts 
     Vector bstparts = new Vector();
        
     if(parts != null) {
        for (WSEncryptionPart part : (Vector)parts)
        {
          if(part.getName().equalsIgnoreCase("Body")) {
           // add the BODY element as by default if signature parts is empty
           // it signs the BODY element.
            bstparts.add(part);
          }
        }
     }
        
     // add the empty signature to the Security Header
     addReferencesToSign(bstparts, secHeader);
     // prepend the signature at the top of the Security Header
     prependToHeader(secHeader);
     // compute the digest values for the BODY element signature using the 
     // BinarySecurityToken 
     computeSignature();
        
     if (bstToken != null) {
        // prepend the BinarySecurityToken element at the top of the signature in 
        // Security Header
        prependBSTElementToHeader(secHeader);
     }
        
     // continue with the normal process of signing and adding IssuerSerial signatures
     prepare(doc, cr, secHeader);
     SOAPConstants soapConstants = 
     WSSecurityUtil.getSOAPConstants(doc.getDocumentElement());

     if (parts == null) {
        parts = new Vector();
        WSEncryptionPart encP = 
            new WSEncryptionPart(
                soapConstants.getBodyQName().getLocalPart(), 
                soapConstants.getEnvelopeURI(), 
                "Content"
            );
        parts.add(encP);
     }

     addReferencesToSign(parts, secHeader);
     prependToHeader(secHeader);
     // Eliminate call to prependBSTElementToHeader() as it is called beforehand
     computeSignature();

     return doc;
}
     ............ 
}

Receiving Response with Security Confirmation:

Initially “enableSignatureConfirmation” was set to “true” only in the wss4jInConfiguration.

 
               ......................
               
                       
               
               ......................
 

This caused the following error to pop up:

0000001e SystemErr     R Caused by: org.apache.ws.security.WSSecurityException:
 WSHandler:  Check Signature confirmation: got SC element, but no matching SV
       at org.apache.ws.security.handler.WSHandler.checkSignatureConfirmation(WSHandler.java:392)
       at org.apache.cxf.ws.security.wss4j.WSS4JInInterceptor.handleMessage(WSS4JInInterceptor.java:224)

After repeated combinations and retries it became clear that “enableSignatureConfirmation” has to be set “true” not only for the Wss4jInInterceptor but for both WSS4JOutInterceptors.  The reason predicted is that, there are two “SecurityConfirmation” elements added in the Response from the Weblogic Server. Now at the receiver end, when we enable the “enableSignatureConfirmation” entry in Wss4jInInterceptor, it tries to check for the Security Vector if there are similar two entries in order to verify the corresponding incoming two elements. As both the WSS4JOutInterceptors didn’t  enable the “enableSignatureConfirmation” entry, there are no entries in the Security Vector to verify. Hence we get the above exception.

......................
<wsse11:SecurityConfirmation>sdfsa9er8sd9f8sd9fgds</wsse11:SecurityConfirmation>
<wsse11:SecurityConfirmation>sdfsa9er8sd9f8sd9fgds</wsse11:SecurityConfirmation>
......................

                           
               ....................
               
                       
               
               ....................
       

Further when tried to alter the contents of even one of the element, the same error as below is thrown again as the contents of the SC elements don’t match with the contents in the SC Vectors.

org.apache.cxf.binding.soap.SoapFault: WSHandler: Check Signature confirmation: got a SC element, but no stored SV.

Going further when the value of the “action” entry was “Timestamp Signature” it threw the following exception:

Security processing failed (actions mismatch)
Caused by: org.apache.ws.security.WSSecurityException: An error was discovered processing the <wsse:Security> header
at org.apache.cxf.ws.security.wss4j.WSS4JInInterceptor.handleMessage(WSS4JInInterceptor.java:290)


After debugging the source it was found that the exception originated from line number 290 of the class org.apache.cxf.ws.security.wss4j.WSS4JInInterceptor. Following was the piece of the code:

// now check the security actions: do they match, in any order?

  if (!ignoreActions && !checkReceiverResultsAnyOrder(wsResult, actions)) {
      LOG.warning("Security processing failed (actions mismatch)");
      throw new WSSecurityException(WSSecurityException.INVALID_SECURITY);
  }

The call to the checkReceiverResultsAnyOrder() method returned false causing it to throw the WSSecurityException. After deeper look in the source code of the checkReceiverResultsAnyOrder() method in org.apache.ws.security.handler.WSHandler class it was found that it compares the Elements in the response with the Actions specified in the configuration entry of WSS4JInInterceptor. It checks whether the same actions are specified corresponding to the elements present in the <SecurityHeader>  of the response. But from the line highlighted in red below, <SecurityConfirmation> and <BinarySecurityToken> elements in the response doesn’t need to have the corresponding Action name in the configuration. This seems logical as the possible values for the “action” entry in the configuration are { NoSecurity , UsernameToken , UsernameTokenNoPassword , SAMLTokenUnsigned , SAMLTokenSigned , Signature , Encrypt , Timestamp , UsernameTokenSignature }.

protected boolean checkReceiverResultsAnyOrder(Vector wsResult, Vector actions);

  java.util.List recordedActions = new Vector(actions.size());

        for (int i = 0; i < actions.size(); i++) {
 
           Integer action = (Integer)actions.get(i); 
           recordedActions.add(action);
        }
 
        for (int i = 0; i < wsResult.size(); i++) {
 
          final Integer actInt = (Integer) ((WSSecurityEngineResult) wsResult
                     .get(i)).get(WSSecurityEngineResult.TAG_ACTION);

          int act = actInt.intValue();

          if (act == WSConstants.SC || act == WSConstants.BST) {
             continue;
          }
 
          if (!recordedActions.remove(actInt)) {
             return false;
          }
        }
 
         if (!recordedActions.isEmpty()) {
             return false;
         }
         return true;
     }

Now looking at the response below from the WS Security enabled Weblogic server, the possible values for the action should be corresponded with , , . But using the above information, there is no such action as “enableSignatureConfirmation” and hence we are left with only the “Timestamp” action in the WS Security configuration entry resolving the exception.

   
      
         
         
         
           2012-03-02T19:35:55Z
           2012-03-02T19:36:55Z 
         
      
   
   
      
         100
         SUCCESS
      
   

When a request is sent to the Oracle Weblogic Server at last the following error was encountered:

WSDLException (at /con:soapui-project): faultCode=INVALID_WSDL: Expected element '{http://schemas.xmlsoap.org/wsdl/}definitions' when trying to load.

On carefully looking the request sent by the Client with the one sent by SOAP-UI, it was found that the Request tag for Client Request was "<CheckEligibility>" while the Request Type of SOAP-UI was "<CheckEligibility Request>".

Windows Commands

   Over the recent years there are many new commands introduced in windows operating systems besides the original DOS commands. These newly added commands enable us to carry out operations which are quite helpful and sophisticated. The full documentations of all the commands is available on microsoft's msdn website.

---

1) XCOPY:
    Following MS-DOS command copies files and directories from source to destination and "/E" creates empty directories, "/C" continues even if there is an error, "/H" includes hidden / system files, "/R" overwrites read only files in the destination, "/K" retaining the file attributes, and "/O" ownership / Access control list information, "/Y" avoiding prompting while overwriting the files.

    xcopy source destination /E /C /H /R /K /O /Y

   Following command copies files and directories from source to destination and  "/C" continuing even if there is an error, "/D" copy the file modified dates, "/S" copy files and subdirectories recursively except empty directories, "/H" include hiddern / system files

    xcopy source destination /C /D /S /H

---

2) ROBOCOPY: 

       Robocopy is the very powerful external command to copy files in windows. Following command copies all the files including the empty directories from the given source location to destination,  

    robocopy source destination   /MIR

---

3) TASKKILL:

     It is used to kill one or more tasks / processes using process id or process name. The following command terminates the process by name forcefully.

    taskkill /im processname /f

     The following command on the other hand terminates all the processes running by the use name "john".

     taskkill /F /FI "USERNAME eq john"

---

4) NETSTAT: 

     It displays active TCP connections, ports on which the computer is listening, Ethernet statistics, the IP routing table, IPv4 statistics. The following command displays the process actual file name using the "-b" option.

    netstat -b

---

5) SHUTDOWN:

     The remote shutdown tool enables to shutdown the local or remote computer within the network.

     Following command shuts down the computer by closing all the applications after specified time delay using "/t" option and displaying the message.

     shutdown \\computername /l /a /r /t:xx "msg" /y /c

     shutdown /l /t:120 "The computer is shutting down" /y /c

     Following command reboots "/r" the remote machine specified using "/m" option. It forces all the applications to close after a a minute delay "/t" with the reason "Application: Maintenance (Planned)" and the comment "/c" "Reconfiguring Applications" type:

     shutdown /r /m \\RemoteMachine /t 60 /c "Reconfiguring Applications" /f /d p:4:1

---

6) SCHTASKS:

     Schtasks command is used to query or execute the tasks inside the Task Scheduler.

     Following command lists all the tasks present on the remote machine.

     schtasks /query /s \\RemoteMachine

     Following command lists all the tasks matching the name "MyTask" present on the remote machine.

     schtasks /query /s \\RemoteMachine  | findstr "MyTask"

     Following command runs the specified task name with the full path present on the specified remote machine.

     schtasks /run /s \\RemoteMachine /tn "\Microsoft\Windows\Tasks\MyTask"

     Similarly following command ends the specified task on the remote machine.
     schtasks /end /s \\RemoteMachine /tn "\Microsoft\Windows\Tasks\MyTask"

     Following command queries the task matching the name "\Microsoft\Windows\Tasks\MyTask" present on the remote machine. It displays advance properites of the task in a list format.

     schtasks /query /s \\RemoteMachine /tn "\Microsoft\Windows\Tasks\MyTask" /fo LIST /v

     Also we can create a new task in the task scheduler using the following command:
     schtasks /create /tn task_name       /tr "...\path\task.bat"       /sc daily              /st 10:00:00       /s \\ComputerName       /u username       /p password

---

7) SC:
     The SC command is used to communicate the service controller to manage windows services. It helps to create, update and delete windows service using various options which run as background processes. Note that all the sc command options require a space between the equals sign and the value.

     Following command creates a new window service with the specified name and run the executable specified along with the binpath option.
     sc create "servicename" binpath= "C:\Windows\System32\sample.exe" DisplayName= "Sample Service" start= auto

     Following command delete the windows service with the specified name.
     sc delete servicename

     Below command lists all the windows services on the command line.
     sc queryex type= service state= all | find "_NAME"

     Alternatively following service commands can be used to start/stop windows services:
     Start a service:       net startservice
     Stop a service:       net stopservice
     Pause a service:     net pauseservice
     Resume a service:  net continueservice

---

8) WMIC:
      The WMIC command provides a command line interface to Windows Management Instrumentation (WMI). WMI is the infrastructure to handle data and operations of the windows operating system and enables to carry out administrative tasks using WMI scripts.
   
     Following command gives the hardware architecture details of the CPU of the current machine
     wmic cpu get caption

     Below command provides the information regarding the current Windows OS architecture, primarily 32/64 bit system.
     wmic OS get OSArchitecture

---

9) PSEXEC:
     This is a utility tool which allows us to execute commands on the remote machines redirecting the remote console output to our local system. There are many other advance usages of the tool.

     psexec \\ComputerName cmd

---

8) NET USE:
      The NET USE command enables to connect or disconnect a computer computer from a shared resource, or to display information about computer connections. The below command assigns the disk drive Z: to the shared directory on \\zdshare

     net use Z: \\zdshare\IT\deploy

     The below command disconnects the Z drive from the \\zdshare directory.

     net use Z: /delete

     Help Option: Use the "/?" option to display the help for the command

     net use /?

---

8) FINDSTR:
      The FINDSTR command is used to search for patterns of text in files using regular expressions. Find the specified text "APC" with /c as a literal search string with non case-sensitive search. Also repeat the search for zero or more occurrences of previous character or class.

     findstr /i /c:"APC" *

---

Test Driven Development

Test Driven Development is famous software development process which relies on the developer to write an automated test case before writing any piece of functional code. It emphasizes series of unit tests and re-factoring to provide a simple design.

   Everyone is accustomed to the general practice of software development which looks as below:

• Design: Figure out how you're going to accomplish all the functionality.
• Code: Type in the code that implements the design.
• Test: Run the code a couple of times to see if it works, then hand it over to QA.

On the other hand Test Driven Development modifies this approach as below:

• Test: Figure out what the next chunk of function is all about.
• Code: Make it do that.
• Design: Make it do that excellently.

As described above TDD completely inverts the accepted ordering of 'design-code-test'. So, from one view, TDD just puts the design after the test and the code. Refactoring is considered as pure design in TDD.

   In TDD world we are not allowed to figure out a complete or excellent design to get our test (and all existing tests) to pass, before we start coding it. Although there is sometimes a debate on whether there should be some kind of initial design phase were interfaces (along with methods signature) for the future classes needs to be defined. Further it is not allowed to reduce or skip the "refactor" step during the TDD development. Hence after each iteration of passing test, there should be refactoring done on the code which indirectly contributes to the design. Also once a test is written, TDD allows us to do either of the following during implementation to pass the test:

1. Reuse some existing code
2. Introduce meaningful new class(es) and method(s)
3. Copy existing method(s) and change the copies

TDD helps in certain aspects of the integration, as the entire process a divided into a series of small steps. The more often we check in the code in version control system, and the smaller our changes are, the less likelihood of getting any 'merge conflicts' with others. Also every commit is a guaranteed fallback position, a piton in the rock that we can easily go back to if we slip and fall.

Below is the Red-Green-Refactor Rule for Test Driven Development:

RED	When you write the test, you are designing the behavior you expect the code-under-test to perform.
GREEN	When you write the code to pass the test, you are designing the internal implementation of that behavior.
REFACTOR	Your micro-focus on getting to green probably 'un-designed' the code. When you refactor you are re-designing.

The Stepwise Premise for TDD goes as below:
   -  Can gigantic complex architectures really be created using nothing other than red-green-refactor?
   -  Consider these issues:

• All large solutions don't just materialize out of nowhere; they are ultimately created in modest steps anyway.
• Even if we have analysis and design phases for large-scale architectural features, we can still develop using TDD.
• Considerable data is available to support the idea that complex global design processes frequently don't work.
• TDD has a serious track record: it is being used all over the world to create complex systems.

Below are the commonly used TDD patterns:

Specify It

• Essence First: What is the most basic functionality needed, not including anything fancy
• Test First:       What exactly will we be testing? Capture that in the test method name.
• Assert First:    What behavior would you like to check?  Writing the assert statement will lead us to produce the structure backwards by "backfilling the method" by declaring the objects and methods we need to create as well as the expected result of calling the new code.

Frame It

• Frame First: Create whatever class(es), constructor(s) and method(s) are needed by our assert statement.

Evolve It

• Do The Simplest Thing That Could Possibly Work: Focus on minimalism by asking oneself to program only what is absolutely necessary to pass a test.
• Break It To Make It: Write a new test code that we know will fail because as our production code isn't capable of handling the new test.
• Refactor Mercilessly: Make design improvements continuously, aggressively, mercilessly avoiding really bad code.
• Test Driving:  In TDD, we don't want to stray too far from the Green Bar.

Finally, Robert Martin, one of the fanatic devotee of Test Driven Development provides the three laws of TDD in his book Clean Code as below:

• First Law: You may not write production code until you have written a failing unit test.
• Second Law: You may not write more of a unit test than is sufficient to fail, and not compiling is failing.
• Third Law: You may not write more production code than is sufficient to pass the currently failing test.

Refactoring generally involves by taking an existing class that's too complex, and break it into smaller classes, each of which takes part of the old class's responsibility, and both of which work together. There are numerous advantages of refactoring the classes to smaller ones, some listed as as follows:

   1)  By making classes smaller, thus easier to grasp at one time.
   2)  By aligning the smaller classes with a well-understood functional breakdown of the underlying problem.
   3)  By making the couplings between classes mirror the couplings between functionality.
   4)  By (ultimately) allowing complex systems to be built by composing many simpler objects.
   5)  By making each smaller class easier to test.

Refactoring also involves Decremental Development, which means finding ways to shrink the code even as we continue to add new features. All the common functionality are moved as a part of library, while pre-existing libraries (core as well as external) with required implementation is searched for instead of re-inventing the wheel.

GUI Applications

In order to apply TDD on GUI applications, they need to have clear separation between user interface and operational logic most commonly achieved by MVC pattern. Although the model/view split isn't the only technique for TDD'ing GUI's, but it does represent the meta-pattern for all of them.
Following can be achieved by splitting responsibilities:

• We can test the Model by having our TestCase pretend to be the View.
• The most important interactions are on the Model, enabling to test core functionality.
• We can use fake domain objects for testing which are in turn are used by the Model.
• We can test the View by creating a fake Model and driving it that way.
• The View can be tested by driving the window's programmatically.

A lot of enhancements can be applied to the Model-View split further such as follows:
 - Add Publisher-Subscriber to allow multiple Views on the same Model.
 - Add a Controller class to translate View-gestures into Model-commands.
 - Add a Command system to isolate and manipulate individual commands.


Test Driven Development Shortcomings

TDD is a development process which assures quality by enforcing unit tests. Although the quality of the code mainly depends on the quality of tests, not when the tests are written during development or how many lines are covered. The essential purpose for writing unit tests is to reduce the possibly of defects in the development phase itself and provide a set of automated tests to validate future changes without introducing new defects. Although such approach is greatly beneficial, the question raised often is to what extent should the tests be written ? When does this approach looses efficiency over the value of auto-tested code ? Does this provide optimal solution to the complex process of software development and unforeseen defects. Is the time and effort spent in writing unit tests to prevent and decrease defects the best approach ?

Most of the Unit Testing tutorials, TDD books and sites describe the approach with basic examples such as processing students grades, calculating wages etc. Although it does gives us a perspective and seems to make the approach by far the best one, but when applied in the co-operate world, such approach has some inherent issues listed as below:

1) Testing a piece of code completely, may involve huge number of scenarios to be considered. Even to select the subset of critical cases and write the test cases for them, it involves almost similar effort as writing the original functional code. But even after selecting a subset of critical cases, we still open ourselves to the possible defects occurring from the ignored scenarios. How to decide which cases are critical and which should be ignored. Some cases may be ignored before, but considering the entire system, such cases could lead to vital failures. Hypothetically, even if we painstakingly compile all the critical cases and wrote unit tests for the entire application, we cannot be sure that there wouldn't be any defects coming up from the unit tested code. Often times, the unit tests validate obvious scenarios (mostly by replicating the code/object in unit test or verifying if the method does get called) thus providing us with a false sense of security. This mostly is caused when the same person writes both the test and the code.

2) Compared to most of the unit testing examples in tutorials, books and articles, the professional code is not that simple or straight forward to isolate. Many real world systems involves, file handling, calling external services, databases, invoking external processes and multi-threading operations. The outcome of these operations is hard to predict. We cannot comprehend the possible values returned by the external services, or by the database all the times. Some of the scenarios such as concurrent operations, server timeout, etc are difficult to recreate in unit test environment. Even if a unit test could be written to check the handling of possible service failures, it would require a substantial amount of efforts compared to manual or integration testing.

3) The basic premise of TDD is that the test drives the system design and implementation. Hence if the line of code cannot be tested then it shouldn't have be written at all. Sometimes due to the limitations of Unit Testing tools such as Junit, Mockito and others the unit test cannot isolately test a certain piece of code. Static methods is one of such cases were despite using Powermock there are many questions raised over the effectiveness of those tests. Also private class fields/methods mostly tend to be changed to lower access modifiers to facilitate unit testing as far as Junit is concerned. Concerns are also raised about the use of Mockito's InjectMocks in unit tests and recommended to use constructor based auto-wiring instead of setter or field based auto-wiring. This ultimately restricts the usage of some features of the programming language or the frameworks inside the boundaries of testability often tagged as bad design.

5) As mentioned previously by Robert Martin, no production code should be written without the corresponding failing test. This totally ignores the fact that whether the unit test is effective, productive and valuable in catching issues. Further it blurs the line between writing a unit test on the behavior/functionality of the code rather than mapping each line of production code with the corresponding unit test. For example creating a new object, setting values to an object, non-conditional calls to library's void methods, logging etc sure compounds to numerous lines of production code, but they hardly articulate any logic or behavior. Consider the following code below:

Properties properties = new Properties();
properties.setProperty("key", "value");
properties.store(new FileOutputStream("C:/test.properties"), null);

The above code creates a Properties object and uses built-in store method of API to create properties file without any conditional logic. There could be many what if arguments made such as what if the store method is not called or file path is incorrect, or properties are not set or incorrectly set etc which often is a slippery slope. But mandating the existence of a line of code or their order is not the purpose of unit test, but is to make sure an independent chunk of code behaves as intended. Any piece of code which only has a single logical flow and returns same or similar results no matter the input has no concrete behavior. Further, if the code does not provide any behavior by itself or relies on external library methods for its behavior then unit testing such code not only adds to overhead and maintenance but fails to provide any productive feedback to detect real problems.
    Further, mandating TDD during a proof of concept or trial and error to fix a known problem not only increases the development overhead exponentially but also distracts the developer from the core task/problem.

4) Someone has said "the line of code that is fastest to write, that never breaks, that doesn't need maintenance is the line you never have to write". In Test Driven Development, as the unit test drives the development (rather than us choosing the critical methods to unit test), there is a lot more test code involved. Multiple scenarios for the given piece of code may encourage duplicate code unless only a single person works on it. In the co-operate projects such big chunks of test code adds up to the maintenance of the system. Badly written unit tests which often involves hardcoded error strings further consume time/effort to maintain. Fragile tests which generate false failures mostly tend to be ignored even in case of valid errors. Modifying the existing functionality using TDD becomes quite challenging as we need to deal with a mesh of interconnected mock objects and a series of test cases.

 Finally the root issue with TDD is not the effort or time required to write them, but their value compared to the effort i.e. Developer Productivity. TDD is much easier to be applied when the design documents dictates the classes/methods and their functionality beforehand. It also would help if all the possible test cases are listed (usually by testers) for the pre-designed classes.


Was it really Behavior Driven Development ?

Since writing this 2013 blog post, many others have joined to question the effectiveness of TDD. David Heinemeier Hansson, the creator of Ruby on Rails has described TDD as "Test-first fundamentalism is like abstinence-only sex ed: An unrealistic, ineffective morality campaign for self-loathing and shaming". After the blog post Kent Beck put forward his sarcastic defense on TDD which later was followed by conversation with Martin Fowler on whether TDD is Dead. Though the conclusion of the conversation was that TDD is valuable in some contexts, but much disagreement prevailed over the number and type of contexts in which it should be applied. Then in the DevTernity 2017 conference Ian Cooper gave a talk on "TDD, Where Did It All Go Wrong" which was promoted by Uncle Bob Martin. In the talk Cooper pointed out that TDD is being practiced incorrectly since we are focused on testing the implementation details instead of testing the system behavior. Due to this we often write more test code than implementation code. Such implementation driven tests with spaghetti of mocks makes refactoring painful, maintenance a nightmare and decreases the overall development productivity. Developers too often don't understand the intent of such tests and are unable to deduce the system behavior by reading them. Enhancements and re-designs becomes difficult as changing the implementation also requires to change the tests which is long haul process.

TDD is mainly practiced by using 'adding a new method to a class' as trigger to write a test. Such test-case per class approach fails to capture the true ethos for TDD. Adding a new class or method is not the trigger for writing tests. The trigger is implementing a requirement. Write tests to cover the use cases or user stories, not the implementation classes or methods. The system under test is not a class but the exports from a module or its facade. The 'unit' of 'unit testing' here really means module, not a class. A class by itself can be the facade, but many classes are implementation details of the module. Do not write tests for implementation details, these change. Write tests only against the stable contract of the (public) API (which can be within a module). Ian Cooper referenced the first book on TDD, "Test-driven Development: By Example" by Kent Beck and pointed out that Kent has explicitly stated that we need to be testing behavior not the implementation. On page 4 of the book Kent writes "What behavior will we need to produce the revised report? Put another way, what set of tests, when passed, will demonstrate the presence of code we are confident will compute the report correctly ?", which clearly refers to test over behavior not implementation. Kent further states that "When we write a test, we imagine the perfect interface for our operation. We are telling ourselves a story about how the operation will look from the outside. Our story won't always come true, but its better to start from the best-possible application program interface (API) and work backward than to make things complicated, ugly, and 'realistic' from the get-go", which affirms testing API's not implementation methods. The tests should run in isolation from other tests, but not the system under test. The unit of isolation is not the class under test, but the tests themselves. Although tests can and should test several classes working together if that is what is needed to test the behavior. We avoid file system, database, simply because these shared fixture elements prevent us from running in isolation from other tests, or the tests become slow. But if there is no shared fixture problem (one test does not affect another) then its perfectly fine to talk to database (though in-memory) or file system in unit tests. Focusing on methods for testing creates tests which are hard to maintain and code which is difficult to refactor because implementation details are exposed to the tests. Such tests do not capture the behavior we want to preserve and becomes difficult to understand. Refactoring is the process of changing a software system in such a way that it does not alter the external behavior of the code yet improves its internal structure. It is the step were we improve our design/implementation, produce clean code, remove duplication, sanitize code smells and apply design patterns. During refactoring to clean code we should not write new unit tests since we are not introducing new public APIs / classes. Dependency is the key problem in software development at all scales. Dependency between the tests and the code should be eliminated by avoiding mocking. Tests should not depend on implementation details by using Mocks because changing the implementation breaks such tests.  Hence mocks should be avoided at all costs except to isolate the tests on the module boundaries (databases, external services, file systems).