I can't say a lot about the mechanical side of the project; but I can tell you that the computer would have to be replaced with the computer from the 4 cylinder vehicle (which may or may not mate with the V-6 wiring harness), the mass airflow sensors are totally different and would have to be exchanged, but the plugs on those sensors are totally different, which means the underhood wiring harnesses would have to be exchanged; the exhaust system would have to be reworked to bolt up to the engine, the throttle position sensors are different, the cold start injection is different, the vapor storage system is totally different; The air filter element, air filter housing, and intake air ducting is totally different. In other words; this would become a massive can of worms.
Nevertheless, I agree with you that the 3.0 3VZ-FE Toyota V-6 motors were basically junk. If it were my project; I would swap in a 1996 or later 24 valve 3.4 5VZ-FE V-6 from a 4 runner; or a 3.0 1MZ-FE V-6 from a 1995 or later Camry, Avalon or Sienna. Those motors were golden, and bulletproof. There are some parts, including the computer, which would need to be changed; but the results would be far more worthwhile.