EntityMode.DOM4J seems not to be supported any longer!
Session session = SessionFactoryUtil.getSessionFactory().openSession();
session = session.getSession(EntityMode.DOM4J);
That feature was always experimental and has been removed. I would like to get back to allowing such a thing, but it will more be allowing "marshallers" to be passed in. If you feel like working on that, contact the dev team on the hibernate-dev mailing list.