Java 7 Concurrency

Improvements and new concurrency features have been added to Java 7 as part of the JSR 166Y. This topic is part of the Oracle Certified Professional (OCP) Java 7 Certification exam and is a primary objective for the 1Z0-805 Java 7 Upgrade Exam (Beta version was 1Z1-805).

1. Identify potential threading problems


2. Use java.util.concurrent collections

Following have been updated in Java 7:
ForkJoinPool, Phaser, TransferQueue, ConcurrentLinkedDeque, ThreadLocalRandom - provides better performance with less contention than Math.random. e.g. ThreadLocalRandom.current().nextInt(0,100)


3. Use atomic variables and locks

Atomic variables can be defined using atomic types in java.util.concurrent.atomic package. Using atomic variable ensures that operations on that variable are thread safe and no block/method synchronization is required. Examples, AtomicInteger, AtomicLongArray

Instead of synchronized code, Java 7 provides more efficient java.util.concurrent.locks package.


4. Executors and ThreadPools

There are 3 executor interfaces in java.util.concurrent:

• Executor - to launch new tasks
  Using Executor, (new Thread(runnable)).start() can be written as executor.execute(runnable)
  Instead of starting a new thread, an existing thread may be used from the ThreadPool, thus optimizing performance.
• ExecutorService - subinterface of Executor to manage the executor and the service
• ScheduledExecutorService - subinterface of ExecutorService to manage schedule tasks

ThreadPool is a commonly used implementation of the Executor.

5. Use the parallel Fork/Join framework

New framework in Java 7 to take advantage of multiple processors on a machine.

Java 7 NIO.2

Java File I/O (NIO.2) is part of the Oracle Certified Professional (OCP) Java 7 Certification exam.

The basic classes to focus in java.nio.file are:

• Path and Paths
• FileSystem and FileSystems
• Files and FileStore

The 1Z0-805 upgrade exam (Beta exam was 1Z1-805) objectives for NIO.2 are:

1. Use the Path class to operate on file and directory paths

Path object can be created using Paths.get(...) or FileSystem.getPath(...). FileSystem object can be created from FileSystems.

Path has methods to operate on the path: getFileName(), getName(idx), getNameCount(), getParent(), getRoot(), hashCode(), isAbsolute(), iterator(), normalize(), subpath(begin,end), toFile() - returns java.io.File, toString()

Path has methods to operate based on another given Path/String: startsWith, endsWith, relativize, resolve, resolveSibling, comparesTo

Path has 3 ways to convert a path toAbsolutePath(),  toUri(), toRealPath(Link Option). toRealPath(...) returns an absolute path after resolving redundancies and if argument is true then resolving symbolic links as well.

register method can be used to register a Path for a WatchService and returns a WatchKey.


Exam Watch:

• Path is system dependent. Path on Windows and Linux with same directory structure will be different.
• Path representing file/dir may not exist
• Path can be operated on to append, extract pieces, duplicate, or compare, but java.nio.file.File has to be used on the Path for actual file operations

2. Use the Files class to check, delete, copy, or move a file or directory

Files class has a set of static methods to work on a file/dir. Methods in Files are symbolic link aware. 

Simple example to create file using NIO.2:

Charset charset = Charset.forName("US-ASCII");

String str = "test";
try (BufferedWriter writer = Files.newBufferedWriter(myFile, charset)) {
    writer.write(str, 0, str.length());
} catch (IOException ioe) {
    System.err.format("IOException: %s%n", ioe);
}

File I/O Methods complexity diagram: 

[Ref: Java Tutorials, oracle.com]

Check
Check file: Files.exists(Path, LINK_OPTIONS...), Files.notExists(Path, LINK_OPTIONS...)
If both return false then existence can not be determined - possible when access permission is denied


Check if 2 Paths point to same file: Files.isSameFile(Path, Path)


Delete
Files.delete(Path) may throw NoSuchFileException, DirectoryNotEmptyException, or else for permission problems - IOException. 

ExamWatch:
Files.deleteIfExists(Path) will never throw NoSuchFileException 

Copy
syntax:    Files.copy(Path, Path, COPY_OPTIONS...);example: Files.copy( src, target, REPLACE_EXISTING, NOFOLLOW_LINKS);

COPY_OPTIONS (Requires import static java.nio.file.StandardCopyOption.*;):

• REPLACE_EXISTING - will replace existing file or symbolic link (not target of link) and empty directory. If target is existing non empty directory then FileAlreadyExistsException is thrown.
• COPY_ATTRIBUTES - will copy file attributes including file's last modified time
• NOFOLLOW_LINKS - when specified, the symbolic link will be copied and not the target of the link

Similarly, Files.copy(InputStream, Path, COPY_OPTION) and Files.copy(Path, OutputStream) can be used.

Move
syntax:    Files.move(Path, Path, MOVE_OPTIONS...);
example: Files.move( src, target, REPLACE_EXISTING, ATOMIC_MOVE);

MOVE_OPTIONS (Requires import static java.nio.file.StandardCopyOption.*;):

• REPLACE_EXISTING
• ATOMIC_MOVE - exception is thrown if filesystem does not support atomic move

Some specific exceptions extend FileSystemException that has useful methods - getFile(), getMessage(), getReason(), getOtherFile() if any other file was involved.

 

3. Read and change file and directory attributes

Check accessibility with Files: isReadable(Path), isWritable(Path), isExecutable(Path)
Also available in Files:

• size(Path)
• isDirectory(Path, LinkOption), isRegularFile(Path, LinkOption...), isSymbolicLink(Path), isHidden(Path) getLastModifiedTime(Path, LinkOption...), setLastModifiedTime(Path,FileTime)
• getOwner(Path, LinkOption...), setOwner(Path, UserPrincipal)
• getPosixFilePermissions(Path, LinkOption...), setPosixFilePermissions(Path, Set<PosixFilePermission>)
• getAttribute(Path, String, LinkOption...), setAttribute(Path, String, Object, LinkOption...)

For bulk operations, String or Class<A> parameter can identify the attributes to be read in one operation using:
Files.readAttributes(Path, String, LinkOption...) or Files.readAttributes(Path, Class<A>, LinkOption...)

PredefinedViews:  
BasicFileAttributeView, DosFileAttributeView, PosixFileAttributeView, FileOwnerAttributeView, AclFileAttributeView , UserDefinedFileAttributeView – Enables support of metadata that is user defined.
 
4. Recursively access a directory tree

Create a  FileVisitor. Then, either of the following can be used:

• walkFileTree(Path, FileVisitor)
• walkFileTree(Path, Set<FileVisitOption - FOLLOW_LINK>, int levelsToVisit, FileVisitor)

 

5. Find a file by using the PathMatcher class

For FileSystem syntax can be "glob" or "regex"

String pattern = ...;
PathMatcher matcher = FileSystems.getDefault().getPathMatcher(syntax + ":" + pattern);

glob:

*   0 or more char, but does not cross directory. Will match .<file> 

**  0 or more char crossing directories

?   single character of name

[]  single character or a range. Within [], *?\ will match itself.

{}  comma separated subpatterns

 \      escape for special characters


6. Watch a directory for changes by using WatchService

Steps to create a WatchService:

• Create "watcher",  a WatchService for the file system.
  

  WatchService watcher = FileSystems.getDefault().newWatchService();

• Register each dir to be watched with the watcher. When registering specify the type of events for notification. A WatchKey instance is received for each registered directory.
  Path dir = ...;
  

  WatchKey key = dir.register(watcher, ENTRY_DELETE, ENTRY_MODIFY);

• Implement an infinite loop to wait for incoming events. When an event occurs, the key is signaled and placed into the watcher's queue.
• Retrieve the key from the watcher's queue and obtain the file name from the key.
  WatchService provides poll(), poll(long, TimeUnit), take() to get a signalled key. 
• Retrieve each pending event using key.pollEvents() and process using event.kind().
  event.context() returns the file name.
• Reset the key by calling key.reset(). If true resume waiting for events, else break.
• Close the service: The watch service exits when either the thread exits or when it is closed by calling watcher.close().

[Ref] Java Tutorials on File I/O featuring NIO.2

Java 7 launch video on NIO.2 from Oracle Media Network:

Java 7 Language Enhancements

This topic is part of OCP Java 7 Upgrade Exam - 1Z0-805 (Beta exam was 1Z1-805) Objective 1.


Language Enhancements [JSR 334]

Language enhancements were made as part of Project Coin.

1. String in the switch statement

Java 7 allows String object in switch statement and consequently String literals or final String constants in case. String.equals is used to match the case, hence matching is case sensitive.
[Ref]

2. Binary literals, numeric literals with underscores

Integral types - byte, short, int, and long - can be represented as binary numbers in Java 7. 
Examples: 0b010, 0B101
This representation is very similar to the hexadecimal representation using 0x. 
Using binary representation may be useful when dealing with bitmaps.
[Ref] 


It is sometimes difficult to read large numeric literals, requiring counting number of digits or zeros. With Java 7 enhancement, numbers like ten million and fifty thousand (10050000) can be represented as 10_050_000 to make it easier to read for humans.
[Ref]


3. try-with-resources

Instead of developer dealing with exceptions and closing appropriate resources in catch or finally block, Java 7 allows placing such resources within try parenthesis and automatically deals with closing such resources.
[Ref]
Sample:
try (
  Statement stmt = conn.createStatement();
  BufferedWriter bufw = new BufferedWriter(new FileWriter("foo.txt"));
) {
  ...
} catch(Exception e) {
  ...
}

4. Multi-catch in exception statements, 6. More precise rethrow in exceptions

Instead of catching each exception separately and handle them individually, Java 7 allows to catch and handle group of exceptions separated by "|" . Syntax is of the form:
catch (Exception1 | Exception2) {} 

Exam Watch:

• If a catch block handles more than one exception type, then the catch parameter is implicitly final 
• Bytecode generated by compiling a catch block that handles multiple exception types will be smaller (and thus superior) than compiling many catch blocks that handle only one exception type each. A catch block that handles multiple exception types creates no duplication in the bytecode generated by the compiler; the bytecode has no replication of exception handlers.

On rethrow, Java 7 is capable of determining the appropriate exception thrown based on the exceptions that a method throws. Example:


void method1() throws Exception1, Exception2 {
    try {
        ...
    } catch(Exception e) {
        throw e; // Java 7 can determine if Exception1 of 2 was throws here
    }
}


Exam Watch:

• This analysis is disabled if the catch parameter is assigned to another value in the catch block.
• If the catch parameter is assigned to another value, you must specify the exception type Exception in the throws clause of the method declaration.
• When you declare one or more exception types in a catch clause, and rethrow the exception handled by this catch block, the compiler verifies that the type of the rethrown exception meets the following conditions:
1. The try block is able to throw it.
2. There are no other preceding catch blocks that can handle it.
3. It is a subtype or supertype of one of the catch clause's exception parameters.
• You can rethrow an exception that is a supertype of any of the types declared in the throws

[Ref]


5. The diamond operator with generic declarations

Java 7 allows using <> operator on the right side of a new construct, thereby simplifying generic declaration. Example:

List<String> names = new ArrayList<String>();
List<String> names = new ArrayList();  // Compiler generates warning
List<String> names = new ArrayList<>();  // <> should be used for auto type inferencing
<> operator may work when used within a method call, but recommended use is only in declaration.
[Ref] 



Other References:
- Talk/slides from the owner of Project Coin (JSR 334), Joe Darcy

Oracle Certified Professional Java SE 7 Upgrade Exam 1Z0-805

Upgrade Exam
Java SE 7 Features and Enhancements

The beta version of 1Z0-805 was 1z1-805, which ended on Jan 7, 2012.

Notes on 1Z0-805 Objectives:

• Language Enhancements
• Design Patterns
• Describe the JDBC API
• Concurrency 
  
• Localization
• Java File I/O (NIO.2)
  

External tutorials, notes, and references:

• The Java Tutorials [JDBC] [Concurrency] [ Localization ] [NIO.2]
• Design patterns in JDK 
• [Update Mar 2012]: An official, free, objective-wise tutorial is now available here
  

All Exam 1Z0-805 Objectives:

Language Enhancements [JSR 334]
1. String in the switch statement
2. Binary literals, numeric literals with underscores
3. try-with-resources

4. Multi-catch in exception statements
5. The diamond operator with generic declarations
6. More precise rethrow in exceptions   

Design Patterns
1. Design a class using the Singleton design pattern
2. Identify when and how to use composition to solve business problems  
3. Write code to implement the DAO Pattern
4. Design a class that uses the Factory design pattern
5. Database Applications with JDBC


Describe the JDBC API
1. Identify the Java statements required to connect to a database using JDBC
2. The JDBC 4.1 RowSetProvider, RowSetFactory, and new RowSet interfaces   
3. JDBC transactions
4. The proper JDBC API to submit queries and read results from the database.
5. JDBC PreparedStatement and CallableStatement  

Concurrency [JSR 166Y]
1. Identify potential threading problems
2. Use java.util.concurrent collections
3. Use atomic variables and locks
4. Executors and ThreadPools
5. Use the parallel Fork/Join framework

Localization
1. Describe the advantages of localizing an application
2. Define what a locale represents
3. Read and set the locale by using the Locale object
4. Build a resource bundle for each locale
5. Call a resource bundle from an application
6. Select a resource bundle based on locale
7. Format text for localization by using NumberFormat and DateFormat

Java File I/O (NIO.2) [JSR 203]
1. Use the Path class to operate on file and directory paths
2. Use the Files class to check, delete, copy, or move a file or directory
3. Read and change file and directory attributes
4. Recursively access a directory tree
5. Find a file by using the PathMatcher class
6. Watch a directory for changes by using WatchService

ssh_exchange_identification: Connection closed by remote host

Debug

"ssh -vvv" can be used for verbose debugging info to determine the cause of failing ssh.

Possible Solutions

[1] Concurrent SSH connections

The default number of concurrent copnnections allowed by sshd is usually only 10. If higher concurrent ssh are expected, increase the MaxStartups in /etc/ssh/sshd_config

This case it the most likely if some of the ssh from one machine to another works and rest get connection closed by remote host error.

[2] Corrupted SSH Keys

If even a single ssh from one machine to another doesn't work, then the SSH keys could have been corrupted. The best steps to follow in such a case is to do the following:
- Ensure correctness of /etc/hosts.allow and /etc/hosts.deny
- Remove ~/.ssh from both machines
- Start over ssh setup following my other post on SSH without password.


Excellent reference and additional details is available here.

Linux I/O Disk Schedulers

Available options are generally:

[1] Noop - It is a FIFO based simple scheduler

[2] Completely Fair Queuing [cfq]

[3] Anticipatory - older scheduler, now replaced by cfq

[4] Deadline - Tries to guarantee a service time

Check Current Scheduler

All available schedulers are listed and the current scheduler is placed within square brackets:

$ cat /sys/block/${DEV}/queue/scheduler
noop anticipatory [deadline] cfq

Change Scheduler for a Device

As a sample, changing the scheduler to noop:

$ echo noop > /sys/block/${DEV}/queue/scheduler
$ cat /sys/block/${DEV}/queue/scheduler
[noop] anticipatory deadline cfq

This change does not persist across reboots. The echo statement may be added to /etc/rc.local in order to execute it after every reboot.

Change Scheduler for all Disks

Instead of changing scheduler for a particular device, if it is desired to change the scheduler for all disks on the system, then elevator=<scheduler> can be added to the boot options in /etc/grub.conf.

Disable IPv6 in Linux

Before Disabling IPv6


Ensure you do not require ipv6. Some common modules that use ipv6 are:
rdma_cm, ib_addr, bnx2i, cnic

Configuration Changes to Disable IPv6

To disable ipv6 and prevent the module from loading, the following configuration files require changes:

[1] In RHEL 5 / OEL5 and earlier
/etc/modprobe.conf:

alias net-pf-10 off
alias ipv6 off
options ipv6 disable=1

# Will load true instead of ipv6 module

install ipv6 /bin/true

In RHEL 6 / Oracle Linux 6:
/etc/modprobe.conf is deprecated. Instead add a conf file with any name in /etc/modprobe.d, like:
/etc/modprobe.d/disable_ipv6.conf, with the following content:
alias net-pf-10 off
options ipv6 disable=1
install ipv6 /bin/true


[2] /etc/sysconfig/network:
NETWORKING_IPV6=no

Reboot for Changes to Take Effect


Verify that IPv6 Module is Not Loaded


The following command should not return anything:
/sbin/lsmod | grep ipv6