The Currency Switcher doesn't usually have a significant impact on the performance of a site. It does add some overhead for the conversion of products, but it should be barely noticeable, unless the site is running on a very cheap hosting, which allocates few resources.
One of the most common causes of slow performance is the presence of a large number of WooCommerce sesions in the wp_options table. In some cases, WooCommerce doesn't clean up the sessions properly, and it may fill that table with old session data. This could cause the site to slow down significantly. Cleaning up the table from old data usually solves the issue. You can find more information about this issue and how to solve it in the following article: How to clean up wc_sessions in WooCommerce.
Another common cause of slow performance is the presence of a large number of variations. WooCommerce needs to calculate the prices of each variation as soon as the page loads, and that implies that up to three prices (regular, sale and actual price) are calculated, for each variation and for each product. The Currency Switcher and WooCommerce cache the prices to improve performance, yet such calculations are necessary every so often. To speed up the site, you can use a caching plugin that support dynamic caching (see How to add dynamic caching to your site), or reduce the amount of variations. In this second case, you can use a plugin like Extra Product Options to enter the product attributes as options, rather than combinations of variations.