fix some failures to enforce required fields, and some code refactoring