In addition to, or in combination with cross-validation, I recommend model blending. The hidden decision trees methodology that I developed a while back, do just that:

• decision trees on observations that are better handled by decision trees
• regression on observations that are better handled by regression

Each model produces a score, and both scores (regression based or decision tree based) are re-scaled so that the two scores are interchangeable.

See details here:

• State-of-the-Art Machine Learning Automation with HDT
• Jackknife and linear regression in Excel: implementation and compar…
• Jackknife logistic and linear regression for clustering and predict…