and build some hefty fines in for payments into the contract.
Don't worry about stacking late fees, you can leave those quite nominal. But if this is actually going to be for a large sum of money, you can make it an interest-only balloon note without a prepayment penalty. That should curb the fear off loss. Give the guy 5-7 years to toss you interest, and he can refi it down the road with a conventional loan or work with you if he still can't land financing.
However I have a feeling that this is a smaller sum, probably in the <$30k range, since there's not a qualifying property on the land. If that's the case, just strap him in with a simple note.