@Patrick: Sure, if you can change the data model, that will be a very good solution. But, sometimes reality hurts and you’re stuck with a model that you have no control over. I sometimes do find myself in such a situation, unfortunately, especially when I deal with FlexFields (varchar2 columns) in Oracle EBS.
When trying to filter out rows, be aware that there is no guarantee about what order Oracle executes the various conditions. So you might well find that the invalid number error returns down the road when the query plan changes.
Even if you nest the subquery, i.e.
SELECT t1.col1, t2.col2
FROM (SELECT t.col1
FROM table1 t
WHERE filter_out_non_numbers) t1,
WHERE t1.col1 = t2.col1
there is no guarantee that the inner subquery is logically materialized before the join (and the implicit conversion) happens.