Yes, usually you do need permission.
Exceptions
If you have retained the rights to the article. Otherwise, the publisher usually now owns the copyright and only the publisher can grant permission.
The figure is a simple representation of data (a basic bar graph for example), you may not need permission. Data cannot be copyrighted, so non-creative ways of representing the data are generally considered "fair use."
Further information
The UNLV Graduate College guide notes that “Copyright approvals for previously published materials must be obtained and placed in the appendix.”
More information may be available in the UNLV Thesis and Dissertation Guidelines.
