You can take the uptown 1 subway (a local) all the way from Penn Station/34th Street to Columbia (116th Street). The subway ride takes 20 to 25 minutes on the 1.
You can also take either the uptown 2 or 3 subway from Penn Station/34th Street. These are express trains and you will need to get off at 96th Street and transfer to the uptown 1 train to get to 116th Street. This is the faster choice, saving as much as 10 minutes of travel time, but you must remember to transfer to the 1 subway at 96th Street. (There is a 116th Street stop on the 2 and 3 subway, but it is in the middle of Manhattan and about a mile from Columbia.)
Penn Station has many facets. The Long Island Raiload (LIRR), New Jersey Transit and Amtrak all have trains that stop there. Penn Station is also served by two subway lines: the 1, 2 and 3 on the east end and the A, C and E on the west end. The easiest way to Columbia from Penn Station is via the 1, 2, 3 subway line.