Ship size picks the terminal
The North Port passenger terminal by Busan Station opened in 2015 and was built for ferries and cruise ships from 500 tons up to about 100,000 tons. Anything much bigger goes to the Dongsam-dong cruise terminal on Yeongdo, which the city expanded to handle ships of up to 220,000 tons. So on most modern large ships, expect Yeongdo.
From Yeongdo, the city runs a free shuttle that drops passengers in the Nampo-dong area, about 30 minutes away. That is the right place to be dropped: Jagalchi and Gukje markets are both close to the stop, so a market morning needs no other transport.
Staying on Yeongdo instead
Yeongdo itself has one strong half-day option close to the ship. Taejongdae, the clifftop park at the island's southern tip with its white lighthouse and observation deck, is served by the hop-on hop-off bus's Taejongdae route, which has a stop near the cruise terminal and runs about every 40 minutes. To find it, walk out of the port gates, turn left toward the Maritime Museum and go uphill behind the building to the main road. The walk inside the park from the entrance to the lighthouse is longer than it looks, so allow for it.
On money: cards are taken almost everywhere except the cheapest stalls, but most Korean ATMs refuse foreign cards. Look for machines marked Global ATM, found in some subway stations and convenience stores, if you want cash for the market.
Sources: koreajoongangdaily.com, koreajoongangdaily.com, whatsinport.com