What if I have one work item I want private for a public contact or an internal job?

In this case, you can create a second contact (without an email address) and set the appropriate level of privacy for that contact. Any one-off work items you want to remain private can be added to this contact.