Shrink Disk in WSL2

Ensure Hyper-V installed so you have Optimize-VHD

Shutdown WSL2 with 

wsl --shutdown

In powershell PS  as admin cd into wsl2 dir - e.g. 

[user]\AppData\Local\Packages\CanonicalGroupLimited.Ubuntu20.04onWindows_79rhkp1fndgsc\LocalState

Run: Optimize-VHD -Path ext4.vhdx -Mode full

OR - Look at GitHub - okibcn/wslcompact: Compacts the size of the ever-growing WSL vhdx images.

Kill a service that is stuck on "stopping"

In Admin CMD:

sc queryex [serviceName]

This willl give you the PID

Then 

taskkill /f /pid [PID]

Git and core.filemode - Issues on Windows with executable bits in Git repos

 I've seen this where you check out source in WSL2 (Linux) and a file has an executable bit set.

Then you look at it in Windows using something like SourceTree and the file that it not reporting as modified in Linux is reporting a modified in SourceTree.

The issue is the core.filemode that is set in the repo's .git/config file is determined by the Git running on the OS that did the clone - it'll be "true" for Linux, and "false" if you cloned using a Windows Git.

If you want to avoid problems, after cloning in WSL2, edit the filemode value to false and Windows Got will behave.

If you do that, in order to ass executable attributes to a file you need to use git update-index chmod +x as filesystem changes made in WSL2 won't be picked up.

Running Rancher Desktop and WSL2 on Windows

This guide was written by taking a laptop that had been reset to factory conditions.  You may not need to do all of them depending on the starting state of your laptop

Enable Hyper-V

In BIOS make sure Hyper-V support is enabled

Make sure Hyper-V feature is turned on in Windows:

• Use search bar find 'Turn Windows features on or off'
• In the dialog that is displayed, turn on Hyper-V

Update Windows 

Do Windows update to make sure you got the latest version

Install WSL2

If you haven't installed WSL2 before, in a Windows Command Prompt, run as Administrator, run 'wsl --install'.  You will be required to reboot.

Update WSL2

Check WSL2 is up to date.  Do this even if you just installed WSL2.

To do this, in a Windows Command Prompt, run as Administrator, run:

wsl --update

 You may be required to reboot.

Install Ubuntu

If you haven't installed WSL2 before, visit the Windows app store and install Ubuntu version 20.04 

Validate Connection Between WSL2 and Windows

Make sure you can connect from an Ubuntu shell in WSL2 to a process running in Windows by allowing stuff through the firewall.  

First find a port running on Windows by running in a Windows command prompt

netstat -an | find "LISTENING"

Ensure you have telnet installed in Windows - if you don't use 'Turn Windows features on or off' to install 'Telnet Client'

Prove you can connect to the running port on Windows, by running the below in a Windows Command Prompt, replacing [port] with the listening port you found:

telnet localhost [port]

Run the following in a PowerShell Administrator console to open the Windows WSL network to calls from the Linux OS:

New-NetFirewallRule -DisplayName "_WSL" -Direction Inbound -InterfaceAlias "vEthernet (WSL)" -Action Allow

Find the ip address of your 'WSL' network in windows by in a Windows command prompt running 'ipconfig /all', and looking for the IP address assigned to the 'WSL' network

In an Ubuntu shell run the below, replacing [ipAddress] with the address you found above:

nc -v [ipAddress]

You should see a response saying the connection succeeded. If you do not then you either not opened the firewall, got the ip address wrong, or the specified port is listening anymore

Install Rancher Desktop

Download Rancher Desktop from https://rancherdesktop.io  You need at least version 1.6.2.

Install Rancher Desktop, when prompted:

•      Pick dockerd and moby as the containr runtime
•      Pick the version of Kubernetes (K8s) that matches one that you regularly use

Run Rancher Desktop and use the File / Preferences menu item to hook Rancher Desktop up to WSL2.

Restart your machine

Start Rancher Desktop and wait for it to start

Validate Docker Availability in WSL2

Check that you can use docker commands in WSL. In a Ubuntu shell, run:

docker --version

to see that docker CLI is available.  Run

docker ps

to show you can view docker processes running in Rancher Desktop

If you get a permission problem accessing the docker socket when you run the above then you may have to add some permissions.  In which case do the following in an Ubuntu shell:

sudo addgrp docker

sudo adduser $USER docker

sudo chown root:docker /var/run/docker.sock

sudo chmod g+w /var/run/docker.sock

newgrp docker

Try the command again, and you should see the docker processes.

Validate host.docker.internal Works

You can check that host.docker.internal behaves the way you expect by following the steps below.

As per above, in a Windows command prompt, run the following to find a listening Windows port:

netstat -an | find "LISTENING"

 In an Unbuntu shell run the following to get a container shell:

docker run -it busybox

In the container shell run the below, replacing [port] with the port you found

nc -v host.docker.internal [port]

If this doesn't work, it most likely because you rebooted and the Windows firewall is blocking access. To fix, run the following in a PowerShell, and retry afterwards.

New-NetFirewallRule -DisplayName "_WSL" -Direction Inbound -InterfaceAlias "vEthernet (WSL)" -Action Allow

Useful Links

If you have problems then consult FAQs at https://docs.rancherdesktop.io/faq/

WSL2 Related Network Problems

The WSL2 network is placed in the "Public" category by Microsoft (even though its an internal network), which block access.  So by default WSL2 cannot access ports running in Windows.

In particular DNS doesn't work, so you may find that you can't for example run the following from an Ubuntu shell:

ping www.bbc.co.uk

To make things worse, the network is recreated after a reboot, so any firewall changes you add to allow traffic from WSL2 to Windows may not work following the reboot.

To get around this you need to run the following after every reboot in a Adminostrator Powershell:

New-NetFirewallRule -DisplayName "_WSL" -Direction Inbound -InterfaceAlias "vEthernet (WSL)" -Action Allow

Another issue that can occur is that subnet used by WSL2 comes from the 172.x.x.x range, and that conflicts with some routing that is handed out by the VPN client when you connect to some VPNs.

You most likely see this as an intermittent problem with Ubuntu being able to reach the Internet when you are not connected to the VPN, and that stops working as soon as you connect to the VPN.

Its intermittent because the problem only occurs if the IP address Microsoft assigns to the WSL network on reboot clashes with the subnet range used by the VPN.  Sometimes it will, and sometimes it won't and there is no way to force WSL to use a particular 172.x.x.x subnet in Windows 10.

The easiest way to work around this is to add a dummy network interface into Windows that uses the 172.x.x.x subnet.  If you do that, and reboot then WSL2 is forced of that network and picks an address in the 192.168.x.x range instead

Have a look at the comment from https://github.com/microsoft/WSL/issues/4467 from naruco on 31st March for details of what to do:

https://github.com/microsoft/WSL/issues/4467#issuecomment-810894433

Essentially though its the following in a Powershell run as Administrator, followed by a reboot:

New-VMSwitch -SwitchName “DummySwitch” -SwitchType Internal

New-NetIPAddress -IPAddress 172.16.0.1 -PrefixLength 12 -InterfaceAlias “vEthernet (DummySwitch)”

Its worth noting a lot of this is not a problem in Windows 11 as you can set up proper bridged networks for WSL2 there, so you can pick the ip address range and have a network that is not "Public"

WSL2 On Windows 10 Running WSLg

This is now possible with Windows 10 and solves some problems that occur with firewalls and WSL2.

The problems are:

• https://github.com/microsoft/WSL/issues/5256
• https://github.com/microsoft/WSL/issues/4139
• https://github.com/microsoft/WSL/issues/4585

In particular:

New-NetFirewallRule -DisplayName "_WSL" -Direction Inbound -InterfaceAlias "vEthernet (WSL)" -Action Allow

didn't appear to allow DNS, but I had already set up DNS as below which works.

What I did:

1. Follow instructions here - https://github.com/epomatti/wsl2-dns-fix-config to set a DNS server - I am using the internal one for the company that I find using ipconfig /all
2. Update Windows 10 to latest UBR (must be at least 2311)
3. If version is not at least UBR 2311 then it should now be possible to install via Windows Update the latest 22H2 Cumulative Update to get the version up to 10.0.19045.2364.
4. If you can't get to at least UBT 2311 you can try getting KB5020030 from the Microsoft update catalog and install manually, although proceed at your own risk.
5. Try in command prompt wsl –update
6. 4) If that doesn’t work try installing Ubuntu WSL2 from the App Store

Useful Links

Microsoft Update Catalog for KB5020030

Running WSL that was installed from the Microsoft Store results in "Windows version 10.0.19045.2251 does not support the packaged version of ..." - Super User

https://learn.microsoft.com/en-us/windows/wsl/tutorials/gui-apps

Hiberate Performance

 https://vladmihalcea.com/why-you-should-always-use-hibernate-connection-provider_disables_autocommit-for-resource-local-jpa-transactions/

Use dumb-init in docker images

Why

https://blog.phusion.nl/2015/01/20/docker-and-the-pid-1-zombie-reaping-problem/

https://petermalmgren.com/signal-handling-docker/

https://medium.com/@BeNitinAgarwal/an-init-system-inside-the-docker-container-3821ee233f4b#:~:text=The%20init%20process%20is%20responsible,for%20the%20top%2Dmost%20process.

Solution

 https://github.com/Yelp/dumb-init

Useful Article About Windows Containers With Rancher Desktop

 https://jason.agostoni.net/2022/01/27/rancher-desktop-with-both-windows-and-linux-containers/

https://lippertmarkus.com/2021/09/04/containers-without-docker-desktop/

https://devblogs.microsoft.com/commandline/systemd-support-is-now-available-in-wsl/

IIS - Configuring X-ARR-ClientCert for server farms

Looked around for ages for this.

If you are using IIS as a reverse proxy server, you have ARR installed and are using server farms to load balance incoming requests to multiple back end services, the configuration for some things is done at the server farm level rather than at the default server level.

My problem as how to change the name of the HTTP header that contained the client certificate used by IIS when it was doing Mutual TLS authentication.

So - the IIS server has mutual TS configured and client cert required and it terminates the SSL endpoint.

Without  a server farm you can use the configuration editor at the IIS server level to and configure the system.webServer/proxy entry.  There is a setting clientCertHeaderName that has a default value of X-ARR-ClientCert, and you can change it to something else to get the IIS to send the authenticated client cert downstream.

If you use server farms however, you have to set up a URL Rewrite rule at the IIS server level to pass matching incoming requests to the server farm, and in that case, the X-ARR-ClientCert header name is not affected by the system.webServer/proxy setting.

Instead, at the server level you have to use the configuration manager and look at the "webfarms" confiration.  There will be a collection of the configured server farms.  So pick the one that you want to change.

Modify the "applicationRequestRouting.protocol.clientCertHeaderName" property of that server farm to the value you want to get the desired behaviour.

Oracle and ODBC

For some reason on Oracle 12 at least I having trouble getting the ODBC call SQLColumns to work . I was using it as an existance test for a database table, and Oracle kept returning 942 (table does not exist) even though the table was owned by the schema associated with the user that was being used in the ODBC call. It turned out that the user needed the SELECT ANY DICTIONARY system privilege. Not sure why, but granting that to the user allowed me to make the call successfully.

Getting an IIS reverse proxy via ARR to use IIS custom error pages

This took a while to discover!

The problem - you have  IIS configured with ARR and you have rewrite rules set up to pass incoming requests from IIS to a back office machine, e.g. Tomcat.

You would like to leverage IIS to display customer error pages when either IIS can't find a page, or the server behind the reverse proxy can't.

In the first case (404 coming from IIS directly) you can use the "Error Pages" feature of an IIS site (or the defaults set at the server level).  Just double click the "Error Pages" feature and then in the Actions pane on the right, select "Edit Feature Settings".  Pick "Custom error pages" get IIS to server up the files pointed to in the feature list for each status code.

You can then edit the file to or create a new file and point at it instead for a given error code.

However, by default,  IIS assumes that if a request a reverse proxied then the remote server will handle setting the HTML for error scenarios, so if, e.g.  404 is returned from the reverse proxied server, IIS will display that response, and not interject with the custom page.

To change that behaviour, execute the following in an admin command prompt:
%windir%\system32\inetsrv\appcmd.exe set config -section:system.webServer/httpErrors -existingResponse:Replace

Hey presto, the reverse proxied reponse are now replaced as well.

I found this, after much searching here:
http://forums.iis.net/t/1157934.aspx?Web+farms+with+URL+rewrite+and+error+pages

Connecting SqlServer to Oracle

Install Oracle ODAC

ensure the tnsnames.ora on your widnows PATH has the alias you want to connect to

create a linked server using oracle OLE Provider

  When you link the users:

     The datasource is the name of the tns alias

     If you get a username/password issue - check the radio buttons to be sure your current sql server connection is not using a user that you have not mapped

You can then use the export data tool to make a reasonable job at copying data over (does noyt do a good job with triggers)

   

Totally Useful Test For MTU

http://www.experts-exchange.com/Hardware/Networking_Hardware/Firewalls/A_3110-Setting-WAN-MTU-Size-For-Sonicwall-Appliances.html

Run
ping www.google.com -f -l 1500

and reduce 1500 in increments of 100 until it stops saying packets can't be fragmented
then increase again until you find the max number that doesn't give this error
then add 28

Import PEM into .P12

OpenSSL> pkcs12 -export -in certFile.pem -inkey keyFile.key -out p12File.p12 -certfile intermediateCertFile.pem

JCE and Thales nCipher HSM

This was an education in JCE.  The task was to:

1) Get a private key on a nCipher HSM

2) Use the key to produce a pkcs7 detached signature for a some content

3) Get the public certificate corresponding to the key so that it could be supplied to an external party to verify the signature.

Some terminology:

.pem file - a file that contains a base64 encoded private key or public key certificate, or certificate chain.  Often used to pass public certificates / certificate chains around

.p12 file - a file that contains a private key and public certificate.  Encrypted so you need passwords to get access to it.

Some JCE terminology

KeyStore - interface that allows you get access to files like .pems and .p12s.  You load a keystore with a file, and specific its type, which then allows you to do stuff like gernate keys, obtains keys, certificates etc

Certificate - Represents a certificate! :-)

Provider - abstraction for the implementation classes that provide the implementation of the JCE interfaces like KeyStore.  When you want to do s

Some providers, like the nCipher implement the interfaces in such a way that operations are performed on hardware devices.  Others, like the SUN and BouncyCastle providers do stuff in memory.  The former is more secure because the private key used for encryption does not exist outside the hardware device.

The BouncyCastle provider does stuff in memory, but it also has convenience classes that do translation from one format to another.

So the trick is to use the HSM provider to do all the key generation and signing stuff and then BC to do format changes.

See my private testSign git project for examples of signing.

To get a key into the HSM:

java -Dprotect=module -DignorePassphrase=true sun.security.tools.KeyTool -genkey -storetype nCipher.sworld -keyalg RSA -sigalg SHA1withRSA  -keystore d:/temp/neilltest.dat

This will create the .dat file that contains a reference to the private key in the HSM (but not the key itself). The .dat file is then used as a keystore like a normal JKS file, using keytools

You can use keytool to generate a csr to sign a certificate request that generates a cert that can be used to verify signatures created using the private key

http://www.sslshopper.com/article-most-common-java-keytool-keystore-commands.html

If you want to list certificates in a HSM then:
keytool -list -v -providername ncioherKM -storetype ncipher.sworld -keystore neilltest.dat -storepass [whatever it is]

Java trust store, Mule HTTPS how to

Occasionally you need to connect to an HTTPS server using java where the public certificate protecting the host is not siogned by a trusted 3rd party CA.

For example self-signed certificates.

This would be the case for example in Mule where you use the HTTPS connector with no trust store configuration.

In order to get the certificate trusted it has to be imported into the trust store of the java runtime being used.
 That's held in a file [jre]\lib\security\cacerts

There is an open source tool called InstallCert that can be used to import a certificate into the trust store.

I've got a copy locally in my d:\tools folder, but you can get it and compile it if you want.
Useful links about this


This is also useful:
http://www.dekho.com.au/help/32/default.htm?turl=Documents%2Faddingacertificateintothejavacertificatestore.htm

https://code.google.com/p/java-use-examples/source/browse/trunk/src/com/aw/ad/util/InstallCert.java
http://miteff.com/install-cert

http://forums.visokio.com/discussion/1365/ssl-certificate-error-use-your-trusted-certificate

http://stackoverflow.com/questions/373295/digital-certificate-how-to-import-cer-file-in-to-truststore-file-using

Import .p12 file into .jks (assuming the .p12 has a password)

If it doesn't have a passwod use previous post to add one first.

Obtain the name of the alias for the tomcat key in the certificate file using the following command:
keytool -v -list -storetype pkcs12 -keystore FILE_PFX

Next plug in the source file, alias name, new .jks file name keystore password and new alias into this command:
keytool -importkeystore -srckeystore [MY_FILE.p12] -srcstoretype pkcs12
-srcalias [ALIAS_SRC] -destkeystore [MY_KEYSTORE.jks]
-deststoretype jks -deststorepass [PASSWORD_JKS] -destalias [ALIAS_DEST]

Add a password to a .p12 file

openssl
pkcs12 -in [file.p12] -out [file.pem] -info
pkcs12 -export -in [file.pem] -out [fileOut.p12]

Debugging tests in gradle

Found this out after a bunch of hacking around.

Gradle starts tests in a separate process form the main build script process, so if you want to debug them you have to configure the test task with jvm debug args.

For example:

test.doFirst {
    jvmArgs '-Xdebug', '-Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=5005'
}

If you "gradle test" the build process will run and suspend on the test task.You can then attach a remote debugger to see what is going on

Wrapped and Bare and JaxBElement

More web service binding stuff.

I had a wsdl that was declared elements as both nillable and minOccurs="0" (so that the elements were wrapped in JaxBElement)

AND it was of doc/lit wrapped form, but had an input type that was also an output type.  This was causing the parameters to get wrapped in javax.xml.ws.Holder objects.

I managed to turn both off with the following binding file:

also had an input and output param
<jaxws:bindings wsdlLocation="[LOCATION OF WSDL]"
      xmlns:jaxws="http://java.sun.com/xml/ns/jaxws"
      xmlns:xjc="http://java.sun.com/xml/ns/jaxb/xjc"
      xmlns:xs="http://www.w3.org/2001/XMLSchema"
      xmlns:jxb="http://java.sun.com/xml/ns/jaxb"
      xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/">
         <jaxws:enableWrapperStyle>false</jaxws:enableWrapperStyle>

    <jaxws:bindings  node="wsdl:definitions/wsdl:types/xs:schema[@targetNamespace='[NAMESPACE OF WSDL TYPES]">
      <jxb:globalBindings xmlns:jxb="http://java.sun.com/xml/ns/jaxb" xmlns:xs="http://www.w3.org/2001/XMLSchema">
            <xjc:generateElementProperty>false</xjc:generateElementProperty>
      </jxb:globalBindings>  
  </jaxws:bindings>

</jaxws:bindings>

Stopping Chrome complaining about an internal certificate with "Server's certificate is signed using a weak signature algorithm"

http://my.galagzee.com/2012/04/07/chrome-weak-signature-algorithm/

Migrating data using oracle

http://www.oracle-base.com/articles/10g/oracle-data-pump-10g.php

Useful web tools

https://webcheck.me/
www.browserstack.com

HTTP explained
http://www.jmarshall.com/easy/http/

Sql Server Performance Sending Unicode To Non Unicode DBs

http://server.pramati.com/blog/2010/06/02/perfissues-jdbcdrivers-mssqlserver/

Adding Complex SOAP Headers

Good articles on how to add SOAP headers when you are using CXF and the headers are complex XML rather than simple strings.

http://stackoverflow.com/questions/3807922/with-cxf-actually-groovyws-how-do-i-generate-a-soap-header-with-one-child-nod

http://blog.bemoko.com/2010/08/12/groovyws-cxf-and-net-webservices-with-an-authheader-pain/

Wrapped vs Bare in Doc/Lit web services

Best explanation that I have seen to date.  Includes a description of the constraints that need to be applied to wsdl to make it work

http://atmanes.blogspot.co.uk/2005/03/wrapped-documentliteral-convention.html

PermGen In Grails

Found this useful tidbit on Grails Facebook page - how to fix Grails permgen issues

• Hi ,default ram size occupied by jvm is 64MB you can increase it manually by the code given by santosh CATALINA_OPTS="-Xms256m -Xmx1024m -XX:MaxPermSize=256m". configure the above code in tomcat config file.

  Friday at 09:40 ·  ·  1
• 

  Prabhat Roy ‎2)Some time memory leak causes this problem, than abome wont work use profiler and chekout the code , dont include unnecessary jars and check the code module by module remove unneccessary object creation , if required call garbage collection explicitly to kill the useless objects. its bit difficult to debug.

  Friday at 09:44 ·  ·  1
• 

  Prabhat Roy ‎3)(a)Put JDBC driver in common/lib (as tomcat documentation says) and not in WEB-INF/lib
  (b)Don't put commons-logging into WEB-INF/lib since tomcat already bootstraps it

  Friday at 09:46 · 
• 

  Prabhat Roy new class objects get placed into the PermGen and thus occupy an ever increasing amount of space. Regardless of how large you make the PermGen space, it will inevitably top out after enough deployments. What you need to do is take measures to flush the PermGen so that you can stabilize its size. There are two JVM flags which handle this cleaning:
  
  -XX:+CMSPermGenSweepingEnabled
  This setting includes the PermGen in a garbage collection run. By default, the PermGen space is never included in garbage collection (and thus grows without bounds).
  
  -XX:+CMSClassUnloadingEnabled
  This setting tells the PermGen garbage collection sweep to take action on class objects. By default, class objects get an exemption, even when the PermGen space is being visited during a garabage collection.

  Friday at 09:47 · 
• 

  Prabhat Roy finally , in rare condition if nothing works than restart the server :)

  Friday at 09:

JAXBElement and generateElementProperty

This had me perplexed for a while.

http://docs.oracle.com/cd/E19879-01/820-1072/ahiid/index.html

So, if you have for example a  name attribute which is a String and has minOccurs="0" nillable="true" for its schema element then there are 2 representations for "no value":

<person>
<name xsi:nil="true"/>
<age>21</age>
</person>

and:

<person>
<age>21</age>
</person>

Note that an empty String element is not the same:


<person>
<name/>
<age>21</age>
</person>

This can be interpreted as an empty String, "", rather than nil.

With this definition its not possible to marshal that in a way that does't lose information if you go direct to String, because you have to pick a representation when you unmarshal.

That is if you perform the marshalling activities xml -> String -> xml you have to pick which representation of nil to use in the final XML, and you have no information to allow you to determine which representation you came from.  So its possible they may end up different.

That's not good.  So JAXB doesn't do that by default.  Instead it wraps elements like this in JAXBElement rather than String so its xml -> JAXBElement -> xml, and the representation is preserved.

Said another way, you can avoid using JAXBElement if you pick the representation of nil you want for XML, by having minOccurs="0", nillable="false" for example.

Or, you can tell JAXB how to do it when you do wsdl2java by providing a binding file that contains generateElementProperty="false".

Interesting Grails Pugin - Melody

System reporting within your app.

http://grails.org/plugin/grails-melody
http://code.google.com/p/javamelody/

SmartSheet - project Tool

https://www.smartsheet.com

FreeAgent - Interesting accounting package

http://www.freeagent.com/

CloudFlare cheap CDN

www.cloudflare.com

Typefaces and response designs

http://beta.typecastapp.com/
http://responsive.is/

Nice way to make a sequential number look random

From here: http://stackoverflow.com/questions/611915/obscure-encrypt-an-order-number-as-another-number-symmetrical-random-appea/612085#612085

Useful if you want to use a sequential number as the unique reference for something, but want to obfuscate it so that its non-obvious that its a sequence

Pick a 8 or 9 digit number at random, say 839712541. Then, take your order number's binary representation (for this example, I'm not using 2's complement), pad it out to the same number of bits (30), reverse it, and xor the flipped order number and the magic number. For example:

1 = 000000000000000000000000000001

Flip = 100000000000000000000000000000
839712541 = 110010000011001111111100011101
XOR = 010010000011001111111100011101 = 302841629

2 = 000000000000000000000000000010

Flip = 010000000000000000000000000000
839712541 = 110010000011001111111100011101
XOR = 100010000011001111111100011101 = 571277085

Command Prompt From Here In Windows

Shift + RClick on the folder

simples!

TripleDES block encryption and padding

What the various parts of a cipher string such as "DESede/ECB/PKCS5Padding" means:

http://www.informit.com/articles/article.aspx?p=26343&seqNum=4

Useful Free Tools

www.BitBucket.com - free dvcs with git
www.cloudbees.com - free jenkins - cheap clous based test servers
http://www.rallydev.com - free agile planning
youtrack - free for public - cheap ($10 / month) private bug tracking

github does it all for $7 month for 1 user

Programmatic configuration of logging Handler for Axis 1.x client

This took ages to work out.

Mostly the web examples involve writing  a WSDD to configure a Handler subclass, but I wanted to create a handler which logged outgoing and incoming SOAP messages via Axis 1.4 and I wanted to programmatically control whether the handler was added to the axis client so that when logging was turned off, there were no unnecessary calls being made.

The code I used was to create a subclass of Axis BasicHandler to do what I wanted (by looking at the example LogHandler class provided by axis: http://kickjava.com/src/org/apache/axis/handlers/LogHandler.java.htm)

Then register it (when the configuration was turned on) using the below.  Note "css" is a service locator object that is created when you use WSDL2Java on the WSDL for the service being connected to.  This is done before you call getPort on the locator to get the service stub.


            SimpleProvider clientConfig = new SimpleProvider();
            AxisClientLogHandler logHandler = new AxisClientLogHandler();
            SimpleChain reqHandler = new SimpleChain();
            SimpleChain respHandler = new SimpleChain();
            reqHandler.addHandler(logHandler);
            respHandler.addHandler(logHandler);
            Handler pivot = new HTTPSender();
            Handler transport = new SimpleTargetedChain(reqHandler, pivot, respHandler);
            clientConfig.deployTransport(HTTPTransport.DEFAULT_TRANSPORT_NAME, transport);

            css.setEngineConfiguration(clientConfig);
            css.setEngine(new AxisClient(clientConfig));
     

Grails + CXF Example - Contract First WSDL

There is an excellent tutorial for doing contract first WSDL with CXF in grails here:
http://docs.codehaus.org/pages/viewpage.action?pageId=85983334

After a bit of mucking about I managed to improve it slightly.

Still couldn't get the cxf simpleBean tag to allow injection of grails services - it works but the resultant service fails with errors like:


[Namespace] [method] was not recognized.  (Does it exist in service WSDL?)</faultstring></soap:Fault></soap:Body></soap:Envelope>"

I think this is because spring / grails is proxying the beans set up in resources.xml so the object passed to simpleServer does not have the annotation necessary to publish the webservice.  Found this:
http://www.techper.net/2009/12/03/cxf-method-not-found-when-further-annotated/

Anyway, the solution is to not use simpleServer and do it yourself in Bootstrap as follows:
1) Inject the grails service that implements the web service into bootstrap
2) start the web service in Bootstrap init:


        System.out.println("Starting Security WebService");
        Object implementor = injectedService
        String address = "/serviceAddress";
        Endpoint.publish(address, implementor);
    }

Git Command - Including Deleting Remote Branch

Handy site to remember git commands:
http://gitref.org

The daddy of refs:
http://progit.org/book

A non obvious command to delete a remote branch.

First do git branch -d [branchName] locally
Then: git push origin :[branchName]

This is based on refspecs - doco for which is can be found here:
http://progit.org/book/ch9-5.html

Basically this is saying push an empty reference to [branchName] on the remote system.  That has the effect of deleting it.

Windows Memory Settings - Know When RAM Or Page File Is The Bottleneck

From http://support.microsoft.com/kb/2267427

Monitoring RAM and Virtual Memory usage
 
Performance Monitor  is the principle tool for monitoring system performance and identifying what the bottleneck really is. To start Performance Monitor, open Control Panel, clickPerformance Information and Tools, click Advanced Tools, and then click Open Performance Monitor.

The following is a summary of some important counters and what they tell you.
 
Memory, Committed Bytes: This is a measure of the demand for virtual memory. It shows how many bytes have been allocated by processes and to which the operating system has committed a RAM page frame or a page slot in the pagefile (or both). As Committed Bytes grows above the available RAM, paging increases, and the amount of the pagefile in use also increases. At some point, paging activity starts to significantly affect perceived performance.
 
Process, Working Set, _Total: This is a measure of the amount of virtual memory in "active" use. It shows how much RAM is required so that the actively used virtual memory for all processes is in RAM. This is always a multiple of 4,096, which is the page size used in Windows. As demand for virtual memory increases above the available RAM, the operating system adjusts the size of virtual memory in the Working Set for a process to optimize the use of available RAM and to minimize paging.
 
Paging File, %pagefile in use: This is a measure of how much of the pagefile is actually being used. This is the counter you should use to determine whether the pagefile is an appropriate size. If this counter reaches 100, the pagefile is completely full and operations stop working. Depending on the volatility of your workload, you probably want to set the pagefile large enough so that no more than 50 to 75 percent of it is used. If a large part of the pagefile is in use, having more than one pagefile on different physical disks may improve performance. 
 
Memory, Pages/Sec: This is one of the most misunderstood measures. A high value for this counter does not necessarily indicatey that your performance bottleneck is a shortage of RAM. The operating system uses the paging system for purposes other than for swapping pages due to memory over-commitment.
 
Memory, Pages Output/Sec: This shows how many virtual memory pages were written to the pagefile to free RAM page frames for other purposes each second. This is the best counter to monitor if you suspect that paging is your performance bottleneck. Even if the Committed Bytes value is greater than the installed RAM, a Pages Output/sec value that is low or zero most of the time indicates that there is not a significant performance problem that is caused by not enough RAM.
 
Memory, Cache Bytes
Memory, Pool Nonpaged Bytes
Memory, Pool Paged Bytes
Memory, System Code Total Bytes
Memory, System Driver Total Bytes

The sum of these counters is a measure of how much of the 2 GB of the shared part of the 4 GB virtual address space is actually in use. Use these counters to determine whether your system is reaching one of the architectural limits discussed above.
 
Memory, Available MBytes: This measures how much RAM is available to satisfy demands for virtual memory (either new allocations, or for restoring a page from the pagefile). When RAM is in short supply (for example, Committed Bytes is greater than installed RAM), the operating system tries  to keep a certain fraction of installed RAM available for immediate use by copying virtual memory pages that are not in active use to the pagefile. For this reason, this counter will not reach zero. Therefore, it is not necessarily a good indication of whether your system is short of RAM.

Tomcat clustering

Excellent article:
http://www.tomcatexpert.com/blog/2010/07/20/basic-tomcat-clustering-grails-applications

Apoache Ab For Performace Tests

http://httpd.apache.org/docs/2.0/programs/ab.html

common script:

ab -c 50 -n 1000 -e "grails_list_test.csv" http://localhost:8080/benchmark/book/inzsertData?rows=100

Enabling JConsole JMX Remote Monitoring of Tomcat

1) Make sure your CATALINA_HOME environment variable is set to the home directory of your catalina installation

2) Create a file runCatalina.bat somewhere hat looks like:

@startlocal
SET JAVA_OPTS= -Dcom.sun.management.jmxremote
SET JAVA_OPTS=%JAVA_OPTS% -Dcom.sun.management.jmxremote.port=8086
SET JAVA_OPTS=%JAVA_OPTS% -Dcom.sun.management.jmxremote.ssl=false
SET JAVA_OPTS=%JAVA_OPTS% -Dcom.sun.management.jmxremote.authenticate=false
SET JAVA_OPTS=%JAVA_OPTS% -XX:PermSize=256m -XX:MaxPermSize=256m
SET JAVA_OPTS=%JAVA_OPTS% -server -Xms1024m -Xmx1024m
%CATALINA_HOME%\bin\catalina run
endlocal

3) If running Vista check this out, and apply the bugfix:
http://marxsoftware.blogspot.com/2008/01/making-jps-and-jconsole-work-with-java.html

I.e. need to make sure that "Everybody" has appropriate permission on the folder that JConsole uses

4) Run Catalina

5) Run jconsole (in your %JAVA_HOME%\bin directory).

6) Connect to the tomcat instancer remotely using hostname and 8086 (as defined above)

Instructions on what JConsole output means:
http://java.sun.com/javase/6/webnotes/trouble/TSG-VM/html/p1.html

WSUS and Group Policy

To force client to windows update via WSUS server
wuauclt /detectnow

To  force group policy to be applied
wpupdate /force

Word Useful

1) Alt f9 to toggle field codes.
2) field codes delimited by {}, but you have to menu Insert Field to get them in.
3) custom doc properties via {DocProperty "custom field name"}.

4) Margin height determines where the distinct part of the document starts and ends.  This fixes header and footer height.  Header and Footer determines how far from the edge of the document the header and the footer start.

5) For borders, you can set a point setting to change the distance between a border a and the text in the box

Things To Check Out

Things to check out:

Reverse AJax for scalable AJAX

http://atmosphere.java.net/

http://en.wikipedia.org/wiki/Comet_(programming)

Spring Integration

http://www.springsource.org/spring-integration

Jabber

JSON

GDoc

Canoo ULC

Gradle

Geb

Gaelyic

http://codemonkeyism.com/

Smalltalk Useful

Smalltalk Start Settings

http://www.instantiations.com/docs/FAQ/wwhelp/wwhimpl/js/html/wwhelp.htm#href=va03001.html

[abt.exe] -sp:blparams.in -i icxFile -mo128000000 -mn2000000 -mf16000000 -ml80000000 -mi8000000

-mfXXXX

This option sets the size of fixed space to XXXX bytes. The default is the size of the fixed space in the image when it was saved. In a newly packaged image, the default size is the actual byte size required by the image, plus a small amount.

-miXXXX

This option sets the segment increment for allocation from the operating system to XXXX bytes. When the executable is out of memory, a new segment of XXXX bytes is allocated from the operating system. The default allocation size is 2048K.

-mlXXXX

This option sets the minimum size of free memory after a global garbage collection to XXXX bytes. The garbage collector uses this value to determine if it needs to allocate additional memory or return memory to the operating system. You use this option to specify a minimum amount of oldspace to remain free after a global garbage collection, and thus defer a garbage collection. The default minimum size is 500K.

-mnXXXX

This option sets the size of new space to XXXX bytes. The default is the size of the new space in the image when it was saved. In a newly packaged image, the default size is 256K.

-moXXXX

This option sets the size of the largest old space to XXXX bytes. The default is the size of the old space in the image when it was saved. In a newly packaged image, the default size is the actual byte size required by the image.

Damaged Disk Rescue Tools

Saved My Life

For fixing badly damaged disks / partitions and whatnot
www.diskinternals.com

Useful GUI Mock Up Tools

Pretty cheap and ubiquitous

Also has hooks to confluence and jira - very good.

http://www.balsamiq.com/products/mockups/jira#buy

Free open source

http://pencil.evolus.vn/en-US/Home.aspx

Somore more good ones for jira / Confluence - diagrams in jira/confluence

www.gliffy.com

Grails & Cobertura Bug

Grails & Cobertura - Cobertura does not cover all classes even though there are tests

Found this by accident. It seems like uncompiled classes that are referenced in a Plugin.groovy script are not poperly instrumented when grails test-app --coverage is run.

However if you do "grails compile" then "grails test-app --coverage" everything then works

Wierd but useful to remember!

Useful Tools & Sites

SoapUI - www.soapui.org

Well handy for testing web services. Can import a WDSL and create a test suite.

Burp Suite

Java based proxy for HTTP(S) see what is being sent to a remote localation, manipulate it etc

WireShark

General Network Sniffing Tool - Handy when you aren't sure why a TCP connection is dropping

xSQL

Excellent tool for comparing 2 databases (SQL Server) to document schema differences

Hudson

Continuous Integration



http://www.javapractices.com
http://www.hibernate.org - ORM Mapping
http://www.springframework.org - IOC Server
http://www.quickserver.org/ - Multithreaded network apps
http://maven.apache.org - project building
http://www.theserverside.com - good  j2ee site
http://www.postgres.org - Opensource multiversioning database.  now supports windows
http://www.h2database.com/  Java based DB
http://www.myeclipse.org
http://www.jdocs.org
http://www.javalobby.org
http://cargo.codehaus.org - Container wrapper api.  makes it easier to start stop etc containers from ant

Open SSL

Command to generate a private key
genrsa -des3 -out [CertRoot]/private/privateKey.pem 1024

Command to generate a private key for a self signed cert
genrsa -des3 -out [CertRoot]/private/cAPrivateKey.pem 1024

Command to create a private key for Bank-Link
genrsa -out [CertRoot]/private/banklinkPrivate.pem 1024

Commands to create self seigned cert
req -new -x509 -keyout [CertRoot]/private/cAPrivateKey.pem -out [CertRoot]/ca/CACert.pem -config [CertRoot]/openssl.cnf -days[days]

req -new -x509 -key [CertRoot]/private/cAPrivateKey.pem -out [CertRoot]/ca/CACert.pem -config [CertRoot]/openssl.cnf -days [days]

Command to create a certificate request
req -new -key [CertRoot]/private/privateKey.pem -out [CertRoot]/cert.csr

Command to sign a server certiciate request
x509 -req -in [CertRoot]/cert.csr -CA [CertRoot]/ca/CACert.pem -CAkey [CertRoot]/private/CAPrivateKey.pem -CAserial [CertRoot]/serial -out [CertRoot]/cert.pem -days [days]

ca -policy policy_anything -config [CertRoot]/openssl.cnf -cert [CertRoot]/ca/CACert.pem -keyfile [CertRoot]/private/CAPrivateKey.pem -in [CertRoot]/cert.csr -out [CertRoot]/cert.pem -days [days]

Command to convert a PKCS#12 file to .pem format
openssl pkcs12 -in banklink.pfx -out banklink.pem -nodes

See also - To install cert in IIS
http://www.dylanbeattie.net/docs/openssl_iis_ssl_howto.html

See also - Free SSL supported by browsers
https://cert.startcom.org/

SQL Server - Useful Stuff

To fix orphansed users.
Occurs when you force a db backup onto another instance on another server
------------------------------------------------------------------------------------------------
exec sp_change_users_login 'Auto_Fix', ''

To move data around
You can use SQL Server 2008 to move data into older databases


See where CPU is being used when CPU utilisation is high
---------------------------------------------------------------------
Select
signal_wait_time_ms=sum(signal_wait_time_ms)
, '%signal (cpu) waits' = cast(100.0 * sum(signal_wait_time_ms) / sum (wait_time_ms) as numeric(20,2))
, resource_wait_time_ms=sum(wait_time_ms - signal_wait_time_ms)
, '%resource waits'= cast(100.0 * sum(wait_time_ms - signal_wait_time_ms) / sum (wait_time_ms) as numeric(20,2))
From sys.dm_os_wait_stats


If % signal waits (i.e. time spent waiting for CPU to be free) is greater than 25%, then more CPUs / faster CPUs will help

if % resource waits is high then use the following to see what:
---------------------------------------------------------------------------
SELECT * FROM sys.dm_os_wait_stats

Before doing a check call the following to reset wait counters:
---------------------------------------------------------------------------
DBCC SQLPERF ('sys.dm_os_wait_stats', CLEAR)

How To Get Donmar Warehouse Tickets

Ten seats for every performance are released at 10.30 each morning; there are 20 spaces for standing. Then there are potential returns. If you are prepared to queue from 9.00 on Saturday you can usually get a ticket. Take a book, a coffee, chat to your fellow theatrephiles; it's a risk, but worth it.